Forum Instability

[OnTheFenceDev]

Some of you may have noticed the forum displaying database (mysql) errors on and off over the past few months.

I've been working on identifying the root cause of the problem which appears to be a lack of available memory. This is odd as the server it now resides on has been running a couple of Wordpress blogs for a few years without any significant problems. I guess the forum software is a little more greedy in terms of memory.

I've reviewed the log files and have made some additional changes to the confiruration today so that may well be the end of it - but then again it may not .

I'll keep an eye on things as best I can but i can't guarantee that it won't go down again - bear with me, we'll get there

[LPGC]

Thanks Dave

[OnTheFenceDev]

You may have noticed further disruptions to the forums availability over the past few days - apologies for this.

This morning I put the forum into maintenance mode and performed some database repairs and optimisations - it looks like some of the tables may have been left in an inconsistent state following previous crashes. This doesn't mean that any data has been lost (although I can't be 100% certain that it hasn't) - probably just an indexing issue. Anyway, the server reported those problems so I've now, hopefully, fixed them.

Whether these were what has caused the latest issues or not remains to be seen but the logs are now clear of the previous errors so that's progress at least.

I have a few further avenues to investigate and will continue to try to drive these problems out - worst case scenario is that I spin up a new server and move the forum onto it but I'd rather find and resolve the issue if I can.

Thanks for your patience

[LPGC]

What I have been noticing is that searches seem to take a lot longer than they have in the past. I sometimes reference threads on this forum on other forums etc and to find the correct thread (might have been years ago but I'll remember a keyword from something I posted). Would slow searching also be related to an indexing problem?

[OnTheFenceDev]

It could well be - some of the tables that required attention were related to searches etc.

We've been up for a week now so maybe the issue of stability have been sorted (maybe not) - I'll see if there is anything I can do about the search speed, maybe the forum software has a maintenance routine for these things. I'll take a look

[OnTheFenceDev]

Found the maintenance option but it will probably be quite an intensive process so best left until the quieter times before I run it - one evening I expect.

[OnTheFenceDev]

I've deleted and recreated the search indexes so we'll see how it goes.

The forum has been stable of the past few weeks so maybe my tweaks to the server have sorted it - time will tell I suppose.

[Gilbertd]

I don't suppose it has anything to do with all the spammers and their posts that we keep deleting that is screwing up the databases?

[OnTheFenceDev]

That shouldn't make any difference - I think it's just some overdue maintenance.

On the subject of spam, I'll be looking at some additional plugins shortly which will hopefully put a big dent in that. I'll trial them out on the test site to see what the configuration options are etc and then install the best on on the live site. The proof will be in the pudding really, these things are difficult to test in a controlled environment.

[LPGC]

The search function isn't working

[OnTheFenceDev]

Just checked it (searched Android using the Search in the top right of the site) and it seems fine to me

[LPGC]

It's working now, it wasn't yesterday or the day before (when I posted) or for a few days before that though... it seemed to hang for a minute and then got an error message reading something like 'the system can't handle the request - 500 error'. I've known it do similar previously, and/or searches to take an age to work. Half expected it not to work after reading your post, thought maybe working on Android but I'm on my laptop running W10. All good anyway, working now! Was about to email you if not

Cheers Dave, Simon.

[OnTheFenceDev]

It won't have escaped your attention that the forum is still experiencing some problems

The current hosting server was also serving up my own website as well as one for a client of mine - these were also being taken down by the forum which was a problem. I've taken the opportunity to move those websites to wordpress.com and let them worry about updates etc (it was becoming a pain anyway).

This means that the forum is the only resource running on this server now - but even so it still ran into problems.

Last night I was actively monitoring the systems resources and it was clearly struggling so I have now increased the RAM etc to hopefully help mitigate this. Current indications are that this has helped - but I'm still monitoring the situation as I may well need to increase it further. It seems that the forum requires more resources that the two Wordpress sites that I was hosting on this box.

On that point I noticed that the 'Online user' counts are much, much higher than previously noted. The max user count was in the upper 200's until recently where it entered the 300's. Last night, during the slowdown, there were apparently over 400 connected users. At the time of writing the count stands at 477 with an active count of 282 (higher than the previous, long standing, max count).

I'm wondering why there is such an increase in traffic - is something happening in the world of LPG or is this a malicious denial of service attack (albeit quite low powered)?

I'm continuing to monitor the situation but I'm hoping that the additional RAM will continue to mitigate the instability issues.

[LPGC]

That explains sporadic 'too many connections' errors I was seeing Dave..