Please post all forum bugs here
- Thread starter elgringo14
- Start date
-
Akiba-Online is sponsored by FileJoker.
FileJoker is a required filehost for all new posts and content replies in the Direct Downloads subforums.
Failure to include FileJoker links for Direct Download posts will result in deletion of your posts or worse.
For more information see this thread.
EU systems are back without any package loss...
It's not so easy to find hoster, that will accept everything that is here, AND are stable.
It's not so easy to find hoster, that will accept everything that is here, AND are stable.
When I open new link it takes sometimes more than 10 seconds for page to load. I tested it on two different locations and same issue.
fucked up the certificate replacement 
all working now.
sorry!
all working now.
sorry!
Been there haha. Glad you got it going~fucked up the certificate replacement
all working now.
sorry!
Sorry for the upload issues.
the backup went wrong and pushed all VMs into the grave with IO-errors :/
seems like a proxmox issue with failing backups.
Let's see if we need to change this in the future.
the backup went wrong and pushed all VMs into the grave with IO-errors :/
seems like a proxmox issue with failing backups.
Let's see if we need to change this in the future.
I'm getting a lot of 502 Bad Gateway errors in recent days, usually site goes slow or no response at all for 30-40 seconds up to 1-2 minutes.
Im going to puke on these servers …
The 2nd one worked flawelessly.
I removed the one, who has errors - now the 2nd starts throwing errors …
Going to look into this
The 2nd one worked flawelessly.
I removed the one, who has errors - now the 2nd starts throwing errors …
Going to look into this
Backup fucked up. Only 300kb/s is way too slow. So the typical 1min hiccup was 1h - I stopped it.
I must look into this - new - error <.<
Sorry for the inconvenience
I must look into this - new - error <.<
Sorry for the inconvenience
Still not solved <.<
Disabled the backup until I’m home from Holliday
Also got an error on the filesystem… again …
I hate when that happens
Disabled the backup until I’m home from Holliday
Also got an error on the filesystem… again …
I hate when that happens
so... i fucked up a config & needed to wait till i got to my notebook again.
sorry for the offline-time.
Backup-Issue is not yet solved
sorry for the offline-time.
Backup-Issue is not yet solved
ipv6 offline;
the upstream-provider did have some severe problems with their BGP...
I'm talking with chompy about possible alternatives.
the upstream-provider did have some severe problems with their BGP...
I'm talking with chompy about possible alternatives.
IPv6 is online again.
Please report issues, if there are any.
Europe frontend is still behaving strange sometimes, but there are not that much alternatives. - and they are stable again for the last 2 weeks. (only 2 small hickups)
Please report issues, if there are any.
Europe frontend is still behaving strange sometimes, but there are not that much alternatives. - and they are stable again for the last 2 weeks. (only 2 small hickups)
Database got stuck. The whole system did have 100% load and very high io...
Still investigating what happened.
sorry for the offline time:
Still investigating what happened.
sorry for the offline time:
*sight*
this would have been way to easy, right?
so ... again sorry for the offline-time.
Post-mortem (as of now):
Mysql does have an issue with limits in their query and very large datasets.
There was someone / more than one? That queries all threats of the forum. With the bug of mysql a query did take something like 6 Minutes. After running a few of them, the system maxes out the CPU. (more cores or ram just move the time until it locks again a bit further). The database requsted nearly 1GB/s select queries.
Normally we are idling at ~10-15% of 4 cores and something like 10MB/s read queries.
Timetable:
Identified the issues on the database.
Restart of the database only got us like 1hour of a somehow working system. (see last post)
I tried to update xenforo (since there was an update) -> no success
I tried to update the linux-OS of the main-system, database, redis-cache -> no success
Unfortionatelty the VPN-Connection between the frontend-systems and the backend crashed (not sure why, because all other connections were working; exceopt to the frontend) so I had time to update the firewall, since everything was done anyways.
VPN-Connections did come online again, but the frontend still didn't want to talk with the backend.
so ... time to update the frontends; rebooting them and clearing all cashes.
Still no success...
I tried to upgrade mysql to the newest version -> no success
I tried to go use the old php-versions, because i thought, that this could have been an issue after the first update-attemt. -> nope.
I was 2s away from opening up the replication-nodes for read-access for this query or rolling back all the changes (yay for backups).
Then i started googeling and found the issues with mysql and limited searches on huge datasets ... I should have googled way before all that.
So after that I at least did know where to look to resolv this issue.
There was some plugin for xenforo that already did optimize some of the quries that are affecting us.
So we are now using a plugin, which I hoped, we didn't need. But well...
Since it took me like 20min to write this (lots of people accessing my office today) the systems seem to be stable.
Let's hope for the best.
this would have been way to easy, right?
so ... again sorry for the offline-time.
Post-mortem (as of now):
Mysql does have an issue with limits in their query and very large datasets.
There was someone / more than one? That queries all threats of the forum. With the bug of mysql a query did take something like 6 Minutes. After running a few of them, the system maxes out the CPU. (more cores or ram just move the time until it locks again a bit further). The database requsted nearly 1GB/s select queries.
Normally we are idling at ~10-15% of 4 cores and something like 10MB/s read queries.
Timetable:
Identified the issues on the database.
Restart of the database only got us like 1hour of a somehow working system. (see last post)
I tried to update xenforo (since there was an update) -> no success
I tried to update the linux-OS of the main-system, database, redis-cache -> no success
Unfortionatelty the VPN-Connection between the frontend-systems and the backend crashed (not sure why, because all other connections were working; exceopt to the frontend) so I had time to update the firewall, since everything was done anyways.
VPN-Connections did come online again, but the frontend still didn't want to talk with the backend.
so ... time to update the frontends; rebooting them and clearing all cashes.
Still no success...
I tried to upgrade mysql to the newest version -> no success
I tried to go use the old php-versions, because i thought, that this could have been an issue after the first update-attemt. -> nope.
I was 2s away from opening up the replication-nodes for read-access for this query or rolling back all the changes (yay for backups).
Then i started googeling and found the issues with mysql and limited searches on huge datasets ... I should have googled way before all that.
So after that I at least did know where to look to resolv this issue.
There was some plugin for xenforo that already did optimize some of the quries that are affecting us.
So we are now using a plugin, which I hoped, we didn't need. But well...
Since it took me like 20min to write this (lots of people accessing my office today) the systems seem to be stable.
Let's hope for the best.
