Too many connections ?

[metalslug 36]

(Re-posted here from S&F to this preferred forum category.)

Does this happen because the new & improved site is so wildly popular now ?

Occurred approx 4pm (GMT+11) on 31/10/2019 lasting at least 10 minutes.

[Administrator 53]

Hey there,

It's a bit of an annoyance we've been dealing with related to the MySQL server. We are working with the hosts to determine the cause and fix any future occurrences.

[SethInMI 174]

On 11/1/2019 at 2:39 AM, Administrator said:

t's a bit of an annoyance we've been dealing with related to the MySQL server. We are working with the hosts to determine the cause and fix any future occurrences.

this is still happening. is there a fix planned?

[sfzombie13 324]

i looked into it a little, but couldn't find much specific about it, other than to raise the limit of concurrent connections.  ms sql has a limit of 32767 connections on a windows server, but i couldn't find anything specific on mysql, other than linux servers are limited to 1024 concurrent connections.  honestly, it sounds like a bandwidth issue.  since dz.com lost popularity, it may be that the bandwidth was decreased for some reason (not necessarily by you) and now needs increased, at least during peak times.  that is all just a guess though and i've been wrong before.

[SethInMI 174]

2 hours ago, sfzombie13 said:

honestly, it sounds like a bandwidth issue. 

no it doesn't sound like that to me. bandwidth issues usually result in things running slow, taking too long to load etc. this problem is not like that.

if the result is a database connection issue, then it does seem to be persistent for some time. I don't know how number of database connections relate to how many users are making requests. 

[sfzombie13 324]

i am not that familiar with databases, but have been a network guy for years now.  researching this has led me to find that in mysql, the max number of connections is defaulted at 151, but can be increased to the limit of the os, and linux was listed as having 1024 max.  it didn't give the max for windows and that took a lot to find.  turns out that it depends on the number of cpu cores and amount of ram and that is is theoretically limited to 26+ billion connections but ms sql is limited to 32767 connections, but that isn't mysql.  after researching further, it looks like it may be reaching an os limit if hosted on a linux server instead of a windows server. 

[yoink 321]

I'm getting this almost every day.

 

I do wonder if people accessing the site via mobile and leaving their browser running on their phone is keeping a connection open.

 

Maybe try a timeout option that closes connections if they're inactive?

Edited July 2, 2020 by yoink

[kallend 2,103]

I got the "too many connections" message most of the day yesterday, and it seems that there's a 22 hour gap in posts on every forum, so it seems there's something more going on than too many people being connected.

[wolfriverjoe 1,523]

I got it last night and for a while this morning.

[wmw999 2,515]

I noticed that when I log out of one computer, dz.com doesn't necessarily log me out of any other places where I was logged on. I wonder if people who use dynamic or semi-dynamic IP address locations have countless sessions left out there tying up our connection resources?

I've been out of this world for over 20 years, so please be gentle if this is a ridiculous suggestion.

Wendy P.

[sfzombie13 324]

On 8/13/2020 at 11:12 AM, wmw999 said:

I noticed that when I log out of one computer, dz.com doesn't necessarily log me out of any other places where I was logged on. I wonder if people who use dynamic or semi-dynamic IP address locations have countless sessions left out there tying up our connection resources?

I've been out of this world for over 20 years, so please be gentle if this is a ridiculous suggestion.

Wendy P.

you're close.  you don't have to be logged out when it closes, it just sends the connection specific info back to the server, sometimes the password, sometimes an authentication token, depending on how it's setup.  you are close though, as simply holding a window open can hold the connection, like when you forget to logout of an rdp session and was mentioned above as a dos attack.  like if you have a mobile browser open as someone else above said, it could keep a window open and active, which would cause them to fill up.  it seems to me that with the amount of posts on here, it may have something to do with changing the hosting or limiting the number of connections somehow due to the switch in forum software when they changed up, but it could be the new forum software itself.  l liked the look of the old site way better, that was how i will forever judge forums.

[wolfriverjoe 1,523]

Got this all day today.

There don't seem to be any posts at all during the time I was getting it.
 

[wmw999 2,515]

There never are posts as far as I know. This is a periodic issue, and it seems mysterious to everyone. I know that Meso has looked at it a lot.

Wendy P.

[sfzombie13 324]

On 2/15/2021 at 12:02 PM, wmw999 said:

There never are posts as far as I know. This is a periodic issue, and it seems mysterious to everyone. I know that Meso has looked at it a lot.

Wendy P.

ya know, it could be connections held open by internet scans or something like that.  some sort of a dos attack exploiting a vulnerability in the software that needs patched.  i have done that before with a scanner sending malformed packets that open a connection and hold it when the server is waiting on a response that never comes.  i'd be curious to see a packet capture from the server when it happens. 

[gowlerk 2,245]

The latest example of the site being unavailable lasted about 24 hours. And no one is willing to comment on WTF the problem is. Does no one monitor the site? What makes it come back? If I was buying advertising here I would start reconsidering that decision.
 

Is there anything I can do to help?

[nwt 131]

I think it should be obvious that something is broken here, and it has absolutely nothing to do with how many are using the site or how they are using it.

[gowlerk 2,245]

It went down again this evening. But only for a short time. Is that because someone noticed and fixed it?

 

[JoeWeber 2,808]

On 2/26/2021 at 2:10 AM, gowlerk said:

It went down again this evening. But only for a short time. Is that because someone noticed and fixed it?

 

Inconvenient for us, yes, but such a loss for the greater world is just heartbreaking.

[nwt 131]

On 2/27/2021 at 11:02 AM, JoeWeber said:

Inconvenient for us, yes, but such a loss for the greater world is just heartbreaking.

Wow, this is disappointing. If you want a healthy membership, this attitude won't get you one or help you maintain it.

[kallend 2,103]

It's happened a lot over the past few days.

[nwt 131]

2 hours ago, kallend said:

It's happened a lot over the past few days.

I messaged the administrator a while back to see if he needed some help with this. No response. This place is doomed. Eventually it will just go down and never come back up.

[Meso 38]

  

On 5/3/2021 at 5:31 PM, nwt said:

I messaged the administrator a while back to see if he needed some help with this. No response. This place is doomed. Eventually it will just go down and never come back up.

Apologies, the Administrator account is somewhat redundant as I typically use this account for most of the administrative work. That said, I hopped onto the other account to see your message there. I'll drop you a DM on this account to discuss though.

On the topic of the site going down and never coming back though, that's certainly not the case. The issue is resolved after the mySQL server gets restarted, unfortunately we rely on our host to do that so there may be some delay. But we would like to get this resolved, of course.

[nwt 131]

On 5/6/2021 at 8:44 AM, Meso said:

  

Apologies, the Administrator account is somewhat redundant as I typically use this account for most of the administrative work. That said, I hopped onto the other account to see your message there. I'll drop you a DM on this account to discuss though.

On the topic of the site going down and never coming back though, that's certainly not the case. The issue is resolved after the mySQL server gets restarted, unfortunately we rely on our host to do that so there may be some delay. But we would like to get this resolved, of course.

Glad to hear from you :)

[IanHarrop 42]

Do you have the ability on the host to set a cron task? If so I found this:

To keep things up and running automatically after a crash, I use a simple cronjob to check and restart MySQL if it is down.
Load the crontab editor in the terminal with crontab -e and add the following line:
* * * * * service mysql status > /dev/null || service mysql start
This checks if MySQL is running every minute and redirects stdout to null.

https://mhagemann.medium.com/how-to-auto-restart-mysql-when-it-crashes-during-a-brute-force-attack-d7a03b726b7e

[Administrator 53]

On 5/8/2021 at 5:13 PM, IanHarrop said:

Do you have the ability on the host to set a cron task? If so I found this:

To keep things up and running automatically after a crash, I use a simple cronjob to check and restart MySQL if it is down.
Load the crontab editor in the terminal with crontab -e and add the following line:
* * * * * service mysql status > /dev/null || service mysql start
This checks if MySQL is running every minute and redirects stdout to null.

https://mhagemann.medium.com/how-to-auto-restart-mysql-when-it-crashes-during-a-brute-force-attack-d7a03b726b7e

Unfortunately we're unable to do any individual MySQL work. The hosting company that we use are in the process of transitioning to another company, and they do not offer the ability to setup Crons, nor do they allow one to adjust the max_connections to the MySQL server.

At this point the only way for us to resolve it is to move the entire site to a different hosting provided, which is a bit of a pain. But we're looking into this now, as we do want to ensure there is no downtime from MySQL timeouts on the server side.