WordPress MU

Yeah, we'd definitely have to co-ordinate things. :D

5 seconds isn't a lot is it.... default on apache is 15 seconds.

might be interesting to see what timings a 1k packet ping comes back with.

Not sure how this might help, but I certainly appreciate your efforts. FYI: Tripawds and RVblogz are our two MU installs which choke when I'm logged in to the dashboard.

--- SERVER ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 155.138/159.463/165.122/4.184 ms

--- DOMAIN (tripawds.com) ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 155.632/267.781/485.122/153.710 ms

--- DOMAIN (rvblogz.com) ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 156.586/254.816/437.772/129.487 ms

--- DOMAIN (agreda.com) ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 158.699/265.963/466.718/142.067 ms

--- DOMAIN (liveworkdream.com) ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 144.008/181.969/204.522/27.001 ms

FYI: Before pin-pointing this issue, I did do a ping during a spike and lost major packets.

(I share an office with andrea_r.)

My ping times on those domains were typical for our satellite connection which usually average in the range of 1500-2000 ms.

So are those times for a ping or a ping -s1024?

I get 173 (for 64 bytes) and 198 (for 1032 bytes) from my connection here at home in the UK.

If wpmuguru is getting pings of 2 seconds then its not going to take much for apache to exceed its 5 second keep alive limit. You might also want to check the standard timeout directive as well.

If the server is swamped with Apache connections then loosing packets from a ping is going to happen

That's our normal time for a normal ping. Our latency is very high.

This is all good. Many thanks for everyone's thoughts on the matter!

FYI: From our MotoSat connection...
PING tripawds.com (72.51.35.77): 1024 data bytes
1032 bytes from 72.51.35.77: icmp_seq=0 ttl=51 time=1386.642 ms
1032 bytes from 72.51.35.77: icmp_seq=1 ttl=51 time=1706.202 ms
1032 bytes from 72.51.35.77: icmp_seq=2 ttl=51 time=1735.820 ms
PING tripawds.com (72.51.35.77): 56 data bytes
64 bytes from 72.51.35.77: icmp_seq=0 ttl=51 time=1373.270 ms
64 bytes from 72.51.35.77: icmp_seq=2 ttl=51 time=710.564 ms
64 bytes from 72.51.35.77: icmp_seq=3 ttl=51 time=660.193 ms

@guru: Who is your satellite internet provider and host? I assume you have no such problems. We use a MotoSat/Hughes link (91w) and the box is at ServerBeach.

I'll be tweaking and testing KeepAlive settings to see if it makes any difference. But I keep scratching my head about why this only happens when logged into the wpmu admin. The load remains stable when moderating our forums.

Does MU have any code that would be affected by network conditions? I don't even have to be doing anything in the admin, just visit the dashboard and averages climb.

Thanks again.

sounds like a dos attack but I don't think it is. It could be the myisam database default have you tried to change the defaults to innodb? you could still keep the text table columns myisam. It might help with myisam locking the tables per request. Interesting anyways.

agreda, all I know is that the backend does seem to be a bit heavy. Does it work better if you use a browser with GoogleGears in it?

@dsim85: Thanks! I'll look into that.
@SteveAtty: I've got gears enabled on Firefox, no difference.

It gets more interesting. I've just confirmed this must have something to do with MU. I have no trouble administering our vanilla WordPress blog on the same server. Yet, when I visit the admin of another MU install we have the problem persists.

Also, we bumped the KeepAlive timeout to 15 and the issue seemed worse, with the averages climbing even higher, quicker. I have no problems when accessing the admin via our Verizon MiFi account.

Continues to scratch head... :-/

have you tried to change the defaults to innodb?

I'm trying to figure out how to do this. Is there an SQL query I can run to alter the default structure without affecting existing tables?

I also notice the default collation is latin1_swedish_ci while all tables are utf8_general_ci ... Ditto with the above question for this.

But here's the latest head-scratcher: If I visit the dashboard of any sub-blog as admin, the load averages remain stable. It is only when I go to the primary blog dashboard when they immediately start to climb out of control.

We are in Canada and all residential satellite goes through Telsat. We are also at ServerBeach and we have no issues like yours. We are running NginX with php-fastcgi.

Based on your last post - if you log into the main blog as a subscriber, do you have the same spike?

You could use an sql query, but it would be easier with phpMyAdmin. But this is just a thought only so try it on a development server first. grab a linux distro (if you are using a linux hosting provider) and try and match the setting of your primary server on the linux distro. Ask your hosting provider to see if they can give you thoughs details. If you are being hosted on IIS i cannot help.

if you log into the main blog as a subscriber, do you have the same spike?

Yes, immediately. But only on the primary blog dashboard. I skip over to that subscriber's own blog dashboard and the load averages fall back to normal. :-/

I'd say there's something running (ex. plugin or theme related) on your main blog that's causing the spike. What plugins are you running to generate your home page?

just to put it out there I'm having the exact same mystery load spike issue on Wordpress 2.8.4 and as far as I know it only happens when I'm logged into admin (ie. it hasn't happened when I've been offline, or if it has it's been a temporary spike that didn't require a restart/reboot).

The weird thing about it is that the load can be at 60x plus and I still can load pages and cpanel loads fairly quickly whereas if the server was getting hammered w/ traffic that caused that kind of load it would take an eternity to do either.

so looking into this some more w/ my host the suphp_log file had become 2.5GB, so it appears when it was written to it created the load spike. monitoring top though it does seem load goes up a bit when in the admin, but I think it had more to do w/ time of day/luck that it was only an issue when I was online.

Well, I'm not going to get too excited just yet ... but after disabling Google Gears and thus turning off WP Turbo, I have been able to administer the site while load averages have remained stable. Oh, if it were only this simple!

Another interesting thing though: When logged in as a primary blog administrator (not site-wide admin), the spike did not occur when managing the main blog. It only happened when accessing the Site Admin dashboard as the top site admin.

Will also look into thatsuphp_log idea, keep an eye on things, and report back. Thanks again to everyone for their thoughts. I'm just glad to know now I'm not the only one experiencing such a mystery.

What is your PHP memory limit set to on your server? This may not seem related to the issue you are having, but I've noticed if I have insufficient PHP memory, it'll look like a slow loading issue when it really isn't. This mostly happened to me within Site Admin panels, as well.

You can play around by adding a

define('WP_MEMORY_LIMIT', 'XXM');

definition to your wp_config.php file. Change the XX to a number (like 128).

Just a thought.

I had that problem too Tim, quite some time back (I haven't looked for the thread) and I upped my php memory limit to 128MB.

PHP 5.2.10 memory_limit set to 500M

Still holding steady after turning off Google Gears Firefox add-on! :-)

So what's causing GG to get confused?

Good Question! I'm just happy to at least know what was causing the issue. I can worry about how later.

I can report the exact same problem with gg. when I turned it on and visited my admin page my server load skyrocketed. For now, I am simply going to remove the link in the admin so it's not a choice for the users to select.

And here I thought it was only me, or something to do with our satellite internet connection (since I couldn't replicate when connecting via Verizon). Regardless, I am happy to report we have had no server load issues since disabling Google Gears.

Just in case someone else is poking aroudn, ad few known spikers could also be:

- people putting their own RSS feed in the rss widget
- posts with wp- in the title, which is used in the permalink
- any content links to internal files that it tires to rewrite

We had an issue with missing smiley images and people pasting old posts on different blog hosts. Getting hit with even a couple page vies on another site looking for non-exisitat smilies on *our* site (hotlinked images basically) led to a rewrite loop of epicness.

@anointed Care to share how you disabled the Gears / Turbo dashboard link(s)? Methinks I'd like to play it safe too.

I just tried it on my laptop and turning off Googlegears in my browser makes the whole backend run a lot faster!

Hi , I have the same problem since last week ! I have Wordpress-mu install on Linux web cluster. I am not using google gear or turbo !

For what it's worth, I am seeing the same problem. WPMU 2.9.1.1. Runs fine most of the time. But, we're seeing random, intermittent load spikes where apache seems to spiral out of control Very frustrating.