Forum Instability

Important forum news
Post Reply
Message
Author
OnTheFenceDev
New member
Posts: 23
Joined: Mon Apr 14, 2014 6:33 pm
Location: Devon, UK
Contact:

Forum Instability

#1 Post by OnTheFenceDev » Wed Nov 06, 2019 1:02 pm

Some of you may have noticed the forum displaying database (mysql) errors on and off over the past few months.

I've been working on identifying the root cause of the problem which appears to be a lack of available memory. This is odd as the server it now resides on has been running a couple of Wordpress blogs for a few years without any significant problems. I guess the forum software is a little more greedy in terms of memory.

I've reviewed the log files and have made some additional changes to the confiruration today so that may well be the end of it - but then again it may not :wink: .

I'll keep an eye on things as best I can but i can't guarantee that it won't go down again - bear with me, we'll get there 8)
There are 10 types of people in the world - those who understand binary and those who don't.

LPGC
Installer
Posts: 3916
Joined: Wed Sep 29, 2010 6:01 pm
Location: Yorkshire
Contact:

Re: Forum Instability

#2 Post by LPGC » Wed Nov 06, 2019 1:55 pm

Thanks Dave
Full time LPG installer
Servicing / Diagnostics / Repairs to all systems / DIY conversion kits supplied with thorough tech support
Mid Yorkshire
2 miles A1, 8 miles M62,
http://www.Lpgc.co.uk
Twitter https://twitter.com/AutogasSimon
07816237240

OnTheFenceDev
New member
Posts: 23
Joined: Mon Apr 14, 2014 6:33 pm
Location: Devon, UK
Contact:

Re: Forum Instability

#3 Post by OnTheFenceDev » Sun Nov 10, 2019 11:19 am

You may have noticed further disruptions to the forums availability over the past few days - apologies for this.

This morning I put the forum into maintenance mode and performed some database repairs and optimisations - it looks like some of the tables may have been left in an inconsistent state following previous crashes. This doesn't mean that any data has been lost (although I can't be 100% certain that it hasn't) - probably just an indexing issue. Anyway, the server reported those problems so I've now, hopefully, fixed them.

Whether these were what has caused the latest issues or not remains to be seen but the logs are now clear of the previous errors so that's progress at least.

I have a few further avenues to investigate and will continue to try to drive these problems out - worst case scenario is that I spin up a new server and move the forum onto it but I'd rather find and resolve the issue if I can.

Thanks for your patience :-)
There are 10 types of people in the world - those who understand binary and those who don't.

LPGC
Installer
Posts: 3916
Joined: Wed Sep 29, 2010 6:01 pm
Location: Yorkshire
Contact:

Re: Forum Instability

#4 Post by LPGC » Sun Nov 17, 2019 8:18 pm

What I have been noticing is that searches seem to take a lot longer than they have in the past. I sometimes reference threads on this forum on other forums etc and to find the correct thread (might have been years ago but I'll remember a keyword from something I posted). Would slow searching also be related to an indexing problem?
Full time LPG installer
Servicing / Diagnostics / Repairs to all systems / DIY conversion kits supplied with thorough tech support
Mid Yorkshire
2 miles A1, 8 miles M62,
http://www.Lpgc.co.uk
Twitter https://twitter.com/AutogasSimon
07816237240

OnTheFenceDev
New member
Posts: 23
Joined: Mon Apr 14, 2014 6:33 pm
Location: Devon, UK
Contact:

Re: Forum Instability

#5 Post by OnTheFenceDev » Mon Nov 18, 2019 12:49 pm

It could well be - some of the tables that required attention were related to searches etc.

We've been up for a week now so maybe the issue of stability have been sorted (maybe not) - I'll see if there is anything I can do about the search speed, maybe the forum software has a maintenance routine for these things. I'll take a look
There are 10 types of people in the world - those who understand binary and those who don't.

OnTheFenceDev
New member
Posts: 23
Joined: Mon Apr 14, 2014 6:33 pm
Location: Devon, UK
Contact:

Re: Forum Instability

#6 Post by OnTheFenceDev » Mon Nov 18, 2019 12:59 pm

Found the maintenance option but it will probably be quite an intensive process so best left until the quieter times before I run it - one evening I expect.
There are 10 types of people in the world - those who understand binary and those who don't.

OnTheFenceDev
New member
Posts: 23
Joined: Mon Apr 14, 2014 6:33 pm
Location: Devon, UK
Contact:

Re: Forum Instability

#7 Post by OnTheFenceDev » Sun Nov 24, 2019 9:54 am

I've deleted and recreated the search indexes so we'll see how it goes.

The forum has been stable of the past few weeks so maybe my tweaks to the server have sorted it - time will tell I suppose.
There are 10 types of people in the world - those who understand binary and those who don't.

Gilbertd
Advanced Member
Posts: 8428
Joined: Wed Apr 09, 2008 10:00 am
Location: Peterborough

Re: Forum Instability

#8 Post by Gilbertd » Sun Nov 24, 2019 1:05 pm

I don't suppose it has anything to do with all the spammers and their posts that we keep deleting that is screwing up the databases?
96 Saab 900XS, AEB Leo, sold
93 Range Rover 4.2LSE, Lovato LovEco, sold
97 Range Rover 4.0SE, multipoint, sold
98 Ex-Police Range Rover 4.0, AEB Leo, daily motor
96 Range Rover 4.6HSE Ascot, AEB Leo, my spare


Proud member of the YCHJCYA2PDTHFH club.

OnTheFenceDev
New member
Posts: 23
Joined: Mon Apr 14, 2014 6:33 pm
Location: Devon, UK
Contact:

Re: Forum Instability

#9 Post by OnTheFenceDev » Sun Nov 24, 2019 1:24 pm

That shouldn't make any difference - I think it's just some overdue maintenance.

On the subject of spam, I'll be looking at some additional plugins shortly which will hopefully put a big dent in that. I'll trial them out on the test site to see what the configuration options are etc and then install the best on on the live site. The proof will be in the pudding really, these things are difficult to test in a controlled environment.
There are 10 types of people in the world - those who understand binary and those who don't.

LPGC
Installer
Posts: 3916
Joined: Wed Sep 29, 2010 6:01 pm
Location: Yorkshire
Contact:

Re: Forum Instability

#10 Post by LPGC » Thu Dec 05, 2019 9:15 am

The search function isn't working
Full time LPG installer
Servicing / Diagnostics / Repairs to all systems / DIY conversion kits supplied with thorough tech support
Mid Yorkshire
2 miles A1, 8 miles M62,
http://www.Lpgc.co.uk
Twitter https://twitter.com/AutogasSimon
07816237240

OnTheFenceDev
New member
Posts: 23
Joined: Mon Apr 14, 2014 6:33 pm
Location: Devon, UK
Contact:

Re: Forum Instability

#11 Post by OnTheFenceDev » Sat Dec 07, 2019 6:13 pm

Just checked it (searched Android using the Search in the top right of the site) and it seems fine to me
There are 10 types of people in the world - those who understand binary and those who don't.

LPGC
Installer
Posts: 3916
Joined: Wed Sep 29, 2010 6:01 pm
Location: Yorkshire
Contact:

Re: Forum Instability

#12 Post by LPGC » Sat Dec 07, 2019 9:13 pm

It's working now, it wasn't yesterday or the day before (when I posted) or for a few days before that though... it seemed to hang for a minute and then got an error message reading something like 'the system can't handle the request - 500 error'. I've known it do similar previously, and/or searches to take an age to work. Half expected it not to work after reading your post, thought maybe working on Android but I'm on my laptop running W10. All good anyway, working now! Was about to email you if not ;-)

Cheers Dave, Simon.
Full time LPG installer
Servicing / Diagnostics / Repairs to all systems / DIY conversion kits supplied with thorough tech support
Mid Yorkshire
2 miles A1, 8 miles M62,
http://www.Lpgc.co.uk
Twitter https://twitter.com/AutogasSimon
07816237240

OnTheFenceDev
New member
Posts: 23
Joined: Mon Apr 14, 2014 6:33 pm
Location: Devon, UK
Contact:

Re: Forum Instability

#13 Post by OnTheFenceDev » Thu Jan 16, 2020 10:16 am

It won't have escaped your attention that the forum is still experiencing some problems :-(

The current hosting server was also serving up my own website as well as one for a client of mine - these were also being taken down by the forum which was a problem. I've taken the opportunity to move those websites to wordpress.com and let them worry about updates etc (it was becoming a pain anyway).

This means that the forum is the only resource running on this server now - but even so it still ran into problems.

Last night I was actively monitoring the systems resources and it was clearly struggling so I have now increased the RAM etc to hopefully help mitigate this. Current indications are that this has helped - but I'm still monitoring the situation as I may well need to increase it further. It seems that the forum requires more resources that the two Wordpress sites that I was hosting on this box.

On that point I noticed that the 'Online user' counts are much, much higher than previously noted. The max user count was in the upper 200's until recently where it entered the 300's. Last night, during the slowdown, there were apparently over 400 connected users. At the time of writing the count stands at 477 with an active count of 282 (higher than the previous, long standing, max count).

I'm wondering why there is such an increase in traffic - is something happening in the world of LPG or is this a malicious denial of service attack (albeit quite low powered)?

I'm continuing to monitor the situation but I'm hoping that the additional RAM will continue to mitigate the instability issues.
There are 10 types of people in the world - those who understand binary and those who don't.

LPGC
Installer
Posts: 3916
Joined: Wed Sep 29, 2010 6:01 pm
Location: Yorkshire
Contact:

Re: Forum Instability

#14 Post by LPGC » Thu Jan 16, 2020 11:43 am

That explains sporadic 'too many connections' errors I was seeing Dave..
Full time LPG installer
Servicing / Diagnostics / Repairs to all systems / DIY conversion kits supplied with thorough tech support
Mid Yorkshire
2 miles A1, 8 miles M62,
http://www.Lpgc.co.uk
Twitter https://twitter.com/AutogasSimon
07816237240

Post Reply