AFF pages slow to load - Page 2

**russellw** · 09-03-2013, 06:16 PM

I'd suggest an issue at your end - we've been monitoring the server loads and performance since before the upgrade to see what impact it had and it has substantially improved the average response time compared to a week ago (from 1300 to 600 ms) as well as resolving the query cache low memory prunes issue.

Cheers
Russ

**russellw** · 09-03-2013, 06:23 PM

Just a note - it will be slowish for the next 30 minutes while I rebuild the master indeces.

Cheers
Russ

Jason[98.EL] · 09-03-2013, 07:09 PM

thanks for the work you are doing russ

just a quick one around 7 this morning the site was down ( not sure if it was just my end )
or was it just down

Jason

**DJR-351** · 09-03-2013, 07:21 PM

Quote:

Originally Posted by Jason[98.EL]

thanks for the work you are doing russ

just a quick one around 7 this morning the site was down ( not sure if it was just my end )
or was it just down

Jason

Me to, i couldn't get on at all so you’re not alone.....

+1 on the work Russ is doing.....

**russellw** · 09-03-2013, 09:00 PM

No it was down twice this morning for a few minutes the first time and about 20 minutes the second. Not sure why but I'm reviewing the logs currently to get to the bottom before it happens again!

Cheers
Russ

Geez Louise · 11-03-2013, 11:13 PM

AFF has been painfully slow for me tonight. It has taken me over 2hrs to even be able to login. All other pages are loading fine.

Cheers
Col

foxtrot3 · 12-03-2013, 07:51 AM

Hi. Dreadfully slow for me yesterday as well and got worse by 7.00pm, tried a couple of times after that and it was still going downhill so I gave up and quit at about 10.00pm. Cheers MD

**russellw** · 12-03-2013, 07:55 AM

No reason why that should be the case - looking at the logs from last night the average load times were below 500ms right through until well after midnight they had been around 650 earlier in the evening but even that is quick.

Need to check whether those having issues use the same ISP.

Cheers
Russ

**russellw** · 12-03-2013, 08:04 AM

Also worth noting that I am doing the remaining indices this morning after rebuilding some corrupted database tables that were damaged in the crash Sunday night.

Will have some impact on performance but probably not a lot.

Cheers
Russ

Geez Louise · 12-03-2013, 08:10 AM

I rang my internet provider last night (Optus) and had them reset my modem and check my settings in case it was on my end and they said everything was working fine, however the exchange for my area does seem to be running below it's usual output.

It was just odd that it was only AFF I had an issue with. If our service goes slow for any reason, we generally tend to load a youtube video to see if there is a delay and that was working fine last night.

Cheers
Col

foxtrot3 · 12-03-2013, 08:27 AM

Hi. We use Optus as well but I could load autospeed in the blink of an eye as with google and google images. Cheers MD.
ps: dont you ever sleep Russ. Cheers MD

**MercurySilver** · 12-03-2013, 10:15 AM

adam internet here
everything else was loading fine and quick
aff was much much slower

but its fine this morning
loading up pages very quickly

**supershifty** · 12-03-2013, 10:46 AM

Yesterday evening between 6-7pm (QLD) AAF was very slow for me. Pages would begin to load then stop loading after the first 4-5 posts were displayed, same would happen on refresh. Tried other sites and was ok, did a cold boot and no difference. I'd been logged in most of the day without a hitch. Tried later on at 9pm and all was good. It's flying right now.

I'm not complaining, these things aren't just peculiar to AFF.

Optus cable, IE9, Win7.

Cheers!

**russellw** · 12-03-2013, 05:31 PM

It's odd - if anything it should be somewhat slower at the moment - in fact the average load times are terrible at 1210ms although (thankfully) the slowness is in the apache layer and not the database so we at least fixed that bit!

At about this time we normally have around 500ms in the apache layer and 200ms in the database - the latter having gotten to the point where it was over 700-800 on it's own which is why we did the upgrade.

What I am seeing now is 800ms in apache and 200-250ms of database response so something has now gone awry with the former. I'll do an apache restart shortly (at 6pm) and then watch it for awhile as I suspect a rogue thread or two.

Cheers
Russ

jmac · 12-03-2013, 07:05 PM

Hey Russ have you explored putting nginx as a reverse proxy in front of Apache? Some good improvements to be had.

Sent from my Nexus 4 using Tapatalk 2

Geez Louise · 12-03-2013, 07:07 PM

AFF is now starting to run a little slower as it did last night. I am finding when I hit refresh, or go back to the main page, the top of my address bar just keeps spinning, like it is trying to load the page and hasn't found a finish point.

Perhaps it is just due to the amount of users logged on? Forum appears to be busier from 7-10pm most nights.

Cheers
Col

Bam · 12-03-2013, 07:33 PM

I don't think the number of users online is the problem. It's working fine for me right now with more users online than at times when it has taken minutes to load rather than milliseconds. I gave up at 7 AM this morning. BTW, my ISP is SpinTel using Telstra ADSL2+ infrastructure. Other sites are OK.

**russellw** · 12-03-2013, 07:36 PM

Quote:

Originally Posted by Geez Louise

AFF is now starting to run a little slower as it did last night. I am finding when I hit refresh, or go back to the main page, the top of my address bar just keeps spinning, like it is trying to load the page and hasn't found a finish point.

Perhaps it is just due to the amount of users logged on? Forum appears to be busier from 7-10pm most nights.

Cheers
Col

Whilst 2,052 is reasonably high it isn't massively excessive compared to what we have been seeing. The response times are certainly heading in the wrong direction at the moment - up around 1500ms average which is higher than this time yesterday and even higher than a week ago before the memory upgrade which is counter intuitive.

Quote:

Originally Posted by jmac

Hey Russ have you explored putting nginx as a reverse proxy in front of Apache? Some good improvements to be had.

Sent from my Nexus 4 using Tapatalk 2

I'm not really seeing anything in the apache server stats that give me cause for concern - since the restart:

Server uptime: 1 hour 14 minutes 19 secondsTotal accesses: 179,851 - Total Traffic: 2.8 GBCPU Usage: u2163.59 s16.51 cu2.34 cs0 - 48.9% CPU load40.3 requests/sec - 0.6 MB/second - 16.3 kB/request52 requests currently being processed, 33 idle workers.. all within normal tolerance but having said that the bulk of the load times are still in the PHP layer currently around 900ms average.

Cheers
Russ

**russellw** · 12-03-2013, 07:38 PM

Quote:

Originally Posted by Bam

I don't think the number of users online is the problem. It's working fine for me right now with more users online than at times when it has taken minutes to load rather than milliseconds. I gave up at 7 AM this morning. BTW, my ISP is SpinTel using Telstra ADSL2+ infrastructure. Other sites are OK.

The 7.00 a.m spike this morning was me running the rest of the table re-indexes (well all but the biggest) - spiked the server out to 5000ms response times until it had finished which is why I usually do that stuff in the middle of the night.

Cheers
Russ

**supershifty** · 12-03-2013, 07:55 PM

Quote:

Originally Posted by Geez Louise

It was just odd that it was only AFF I had an issue with. If our service goes slow for any reason, we generally tend to load a youtube video to see if there is a delay and that was working fine last night.

Cheers
Col

When I'm getting errors where a site won't load - ie: site not found, unable to load, 404 ectetera, I go to http://www.websitenotworking.com/ and type in the the URL to get a second opinion.

To check the performance of my connection I go to http://www.speedtest.net/ Once you follow the simple set up the site will remember up to your 10 last tests for comparison - no need to join for the premium package. It's not the be all to end all, just another tool in the box for referencing.

Cheers!

Geez Louise · 12-03-2013, 09:15 PM

Thanks for that supershifty. I have been running speedtests on the hour for the Optus tech's and currently we are running at 76757kbps upload speed and 1338kbps download speed which is within the norm but AFF is still running slow.

I know Russ is working on it and hopefully it will be resolved soon. Currently it is taking me over 20secs to load an AFF page.

I hope some of this information helps you Russ.

Cheers
Col

**russellw** · 12-03-2013, 09:38 PM

Doesn't match what I'm seeing - page loads were high earlier in the evening but have settled down into the mid 500ms bracket as per the chart below.

Apache stats have also settled substantially -

Server uptime: 3 hours 41 minutes 49 seconds
Total accesses: 580,709
Total Traffic: 8.8 GB
CPU Usage: u551.47 s8.86 cu.73 cs0 - 4.22% CPU load
43.6 requests/sec - 0.7 MB/second - 15.9 kB/request
49 requests currently being processed, 33 idle workers

SEZ213 · 12-03-2013, 09:47 PM

Whatever just happened in the last ten minutes - pages are now loading super fast.

Geez Louise · 12-03-2013, 10:22 PM

It is faster but my webpage is still spinning. I seem to have intermittant issues. It is sometimes fast and then slow again.

It is not an issue with the optus network as I just got off the phone to the internet tech and all other pages etc are loading fine.

Cheers
Col

Simple6 · 13-03-2013, 03:43 PM

I gave up this morning trying to get on here on my tablet. All other web pages I was browsing were fine but I could never load AFF gave up after 10mins of white.

**russellw** · 13-03-2013, 08:12 PM

After some tuning last night to take advantage of the additional memory we now have things running at a nice level. Average response times today have been good (including at 0700 when I was on) with a peak (thus far) around 1000ms at 1500 and an overall average of 623ms.

It's still spending way too much time in the PHP serving end (Apache) - not sure if this is caused by compressing pages before they are delivered (which we do to minimise bandwidth) or whether there is something else in Apache that I can do.

At least this is now the reverse of what we had before where more than 70% of the time was taken in the database - it's now 20-25% in the database and the balance serving the PHP.

Realistically I am targeting average loads in the sub 1000ms bracket (which we are doing) but with a near enough 50/50 mix between database and PHP which should result in sub 500ms averages.

For those interested in the details, I've created a little page where the live response times are monitored HERE which shows the rolling 6 hour average and the mix between PHP (cyan) and database (orange) time.

Cheers
Russ

Mr G8 · 13-03-2013, 08:19 PM

Interesting indeed

Geez Louise · 13-03-2013, 09:13 PM

It is running great for me tonight. Thanks for all your efforts Russ. Greatly appreciated

**russellw** · 14-03-2013, 07:20 PM

Mystery solved as to why things have been strange.

In tracking down another issue with thread views not updating, I found that that table was corrupted and there were several hundred temporary tables being created to cache that data which, in turn, was locking the database intermittently.

Hopefully fixed now although we will wait and see what happens next.

Cheers
Russ

ebgtjames · 14-03-2013, 09:52 PM

Not sure if its just me but I cant get any of the smiley things to work, same goes for bold and italic etc...??