The MU forums have moved to WordPress.org

wp-cron causing high server load (2 posts)

  1. iolaire
    Member
    Posted 15 years ago #

    Recently I moved my MU installation (current version) over to Futurehost with a Centos 5 x86 server where I've been experiencing extremely high processor use. I've narrowed it down to wp-cron, when the file is in place my server gets hammered with requests from my own IP, when renamed server load goes back to normal.

    Based on the logs it looks like it is trying to do ping backs on individual posts - I have ~30,000 * 20 sites. I've tried to set my site to close all pingbacks and track backs, but based on the logs it looks like it is trying to pingback all links in each post - which for me are internal links to a click tracking script.

    Is pingbacks on posts an internal setting? Or must it be set per post? If per post can it be set on XMLPC posts?

    Looking at my my sql data I see a null value in the default_pingback_flag, should that be set to "0" or "closed" or does it not need to be set?

    The following are my standard access_logs with wp-cron in place with /outbound/ being a link in a post and :

    [mrsite@mrsite public_html]# tail /home/site/logs/access_log
    xx.xx.xx.xxx - - [19/Oct/2008:21:04:34 -0400] "POST /xmlrpc.php HTTP/1.0" 200 473 "-" "Incutio XML-RPC -- WordPress/2.6.2"
    xx.xx.xx.xxx - - [19/Oct/2008:21:04:34 -0400] "HEAD /wp-content/themes/theme1/images/btn_buynow.gif HTTP/1.1" 500 - "-" "WordPress/2.6.2"
    xx.xx.xx.xxx - - [19/Oct/2008:21:04:34 -0400] "HEAD /outbound/67yv HTTP/1.1" 404 - "-" "WordPress/2.6.2"
    xx.xx.xx.xxx - - [19/Oct/2008:21:04:34 -0400] "HEAD /outbound/52qx HTTP/1.1" 404 - "-" "WordPress/2.6.2"
    xx.xx.xx.xxx - - [19/Oct/2008:21:04:35 -0400] "POST /xmlrpc.php HTTP/1.0" 200 473 "-" "Incutio XML-RPC -- WordPress/2.6.2"
    xx.xx.xx.xxx - - [19/Oct/2008:21:04:35 -0400] "HEAD /outbound/67yv HTTP/1.1" 404 - "-" "WordPress/2.6.2"
    xx.xx.xx.xxx - - [19/Oct/2008:21:04:35 -0400] "HEAD /outbound/52qx HTTP/1.1" 404 - "-" "WordPress/2.6.2"

    the error logs are full of:

    [mrsite@mrsite public_html]# tail /home/site/logs/error_log
    [Sun Oct 19 21:04:25 2008] [error] [client xx.xx.xx.xxx] Request exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace.
  2. iolaire
    Member
    Posted 15 years ago #

    Ok I think I figured this out.
    1. To disable pingbacks and searching posts for enclosures you can do that on a template level
    http://wordpress.org/support/topic/177691?replies=9#post-798828
    -- that allowed me to better look at what was happening without killing the server
    2. Every time a post was searched for enclosures (mp3's and such) it would try to load a missing image along with the other links
    --- this is what was really killing my server, many posts were being searched for enclosures and invalid urls were also being loaded causing redirects

    I'll try turning the search for enclosures back (1.) on tomorrow expecting that things will stay up (since I fixed the image problem in 2.).

About this Topic