The MU forums have moved to WordPress.org

Modifying WP-Cache for WordPress MU (105 posts)

  1. suleiman
    Member
    Posted 16 years ago #

    so you got this working ovidiu?

  2. Ovidiu
    Member
    Posted 16 years ago #

    not really, after my last posts and those changes, I kind of lost interest...
    But go ahead and try those, report back and I'll give it some more tries too, maybe we'll get it working together...

    I don't NEED it I just WANT it ;-) I haven't that much traffic on my wpmu install that would make it a necessity...

  3. suleiman
    Member
    Posted 16 years ago #

    So here's what i've done and it's produced some progress but no real workable results yet:

    in wp-cache.php line 161 change to:

    $wp_cache_config_file = ABSPATH . UPLOADS . '/wp-cache-config.php';

    in wp-cache-phase1.php line 2 change to:

    if( !@include(ABSPATH . UPLOADS . '/wp-cache-config.php') ) {

    and in wp-cache-config-sample.php line 11 change to:

    $cache_path = ABSPATH . UPLOADS . 'cache/';

    This produces the following results:

    A wp-cache-config.php file is placed in each user's $blog_id/files/ folder. And a /cache/ directory is also created in the $blog_id/files/ folder.

    But nothing is written to it because I can't get past this error:

    Warning: fopen(/home/.../public_html/wp-content/blogs.dir/83/files/cache/) [function.fopen]: failed to open stream: Is a directory in /home/.../public_html/wp-content/plugins/wp-cache/wp-cache-phase2.php on line 96

    Almost there though :)

    EDIT: I thought I would include, line 96 of wp-cache-phase2.php is part of a function which reads as follows:

    if ($use_flock)
    $mutex = fopen($cache_path . $mutex_filename, 'w');
    else
    $mutex = sem_get($sem_id, 1, 0644 | IPC_CREAT, 1);
    }

  4. suleiman
    Member
    Posted 16 years ago #

    anyone get farther than i did?

  5. Ovidiu
    Member
    Posted 16 years ago #

    sorry, never had the time to pursue this further :-( but still interested in getting it running

  6. quenting
    Member
    Posted 16 years ago #

    ok, some of my blogs reached 200000 daily PV, so i thought i'd give a shot at trying to make this work.
    I'll pass on the various first changes I made not to have to chmod 777 wp-content, or to go through the "enable" step.
    I have disabled the admin page for users (with the is_site_admin trick).
    I'm not doing the user-specific trick above for the config file, users don't have options, so there's no need for a separate config file, every one has the same.
    Regarding cached file storage, I replaced in the config file:

    $cache_path = ABSPATH . 'wp-content/cache/';

    with

    $cache_path = ABSPATH . 'wp-content/wpcache/'.$blog_id.'/';
    if(!is_dir($cache_path)) {
    mkdir($cache_path);
    }

    wanted thecahce to be dissociated from the object cache for the sake of clarity and being able to purge files separately. The above change will result in 1 directory per blog to be created in the folder wp-content/wpcache/ that will contain the cached files.

    This works at 90%.

    The issues I'm looking into now:
    1) Posting a comment doesn't update the cache. I suspect this is an SK2-related issue
    2) Another SK2-related issue that will show up is that SK2 uses javascript-embedded IPs and codes to validate the comments. If the page is cached, those will be too, resulting in possibly legit comments to be moderated when they shouldn't.
    3) It seems wp-cache embeds some cookie information in the code to store the file. This is normal, because for instance admin and guest do not see the same page. However i need to check how far this goes (do custom cookie values result in a different file created each time ? this could negate the plugin efficiency at 100%).
    4) There are issues with dynamic content in sidebars. If i use categories with counts, the counts aren't updated until the page gets refreshed. It seems that changing the sidebar contents (moving widgets around for instance) does flush the cache though.
    5) There are issues with certain types of plugins. For instance i have a hit counter that records each access to any page. If the page is served from cache, the hit won't be recorded. I need to find a solution for this. The counter output to page is also a problem. Basically i need a way to keep some of the content dynamic within the cached files. it seems possible through the use of <!–mclude --> syntax, however I haven't really understood yet how this works. Also it means that the dynamic functions that are left in the cache must be as much as possible "stand alone", that is, not needing the full wordpress engine to have been loaded to be used, in order to keep the cache useful.

    Feedback welcome.

  7. Ovidiu
    Member
    Posted 16 years ago #

    regarding
    - 1+2 I remember seeing a plugin/module for SK2 that makes SK2 work with wp-cache
    - 5 I never got that weird mclude stuff working, not even on a plain wp install. I never could get it to exclude dynamic content from being cached, I also asked in some forums and never got an answer from someone who got it working..

  8. quenting
    Member
    Posted 16 years ago #

    actually, the blog_id doesn't work, it isn't set at this stage. I have changed the above code to use the blog name (from $_SERVER['HTTP_HOST'] ).

    1) The sk2_wp_cache_plugin indeed solves the problems of comments not passing through.
    2) the plugin doesn't solve that though. The payload plugin seems to be the one messing things. I'm gonna try to find the solution for this, which should be the same as for 4) and 5).
    3) there is no issue with cookies since it only uses wordpress-specific ones to generate the key.

    I was wrong though, changing the sidebar widgets stuff doesn't invalidate the cache. Need to find the right hook to do so. The main problem is most widgets are dynamic. Basically, adding a new page should empty the cache, because of the pages widget not being updated otherwise. Technically, same for comments (latest comment widget), articles, etc. But if the widget isn't used, suddenly there's no need to empty the cache anymore. More complex than at first sight he. Ideally there'd be one file cached per widget and only the appropriate ones would be flushed, but that sounds kinda complex to implement.

    Now looking into that mclude stuff.

  9. quenting
    Member
    Posted 16 years ago #

    to solve the widgets issue, one must do the following:
    in widgets.php, add an action on widgets save:
    find:
    case 'save_widget_order' :
    and after add: do_action('my_save_widget_order');

    Then in wp_cache_phase2, add in the list of add_actions:
    add_action('my_save_widget_order', 'wp_cache_no_postid', 0);

    This should flush the widgets cache and reflect changes in widgets immediately in the blog.

  10. quenting
    Member
    Posted 16 years ago #

    For sk2, I haven't found better yet than to disable the javascript checks when cache was enabled. I know it's not great, I'll try to find something better to do.

  11. lunabyte
    Member
    Posted 16 years ago #

    Here's a possibility for consideration...

    Could the comment area be excluded from being cached entirely? I haven't tried it, and it isn't the "best" solution but still at least serving 80% of a page from cache is better than not serving it from cache at all.

  12. quenting
    Member
    Posted 16 years ago #

    > Could the comment area be excluded from being cached entirely?

    nah, I don't think so. See, the problem is that the most benefit you get from caching is not having to load all the scripts and run all the queries necessary to basic MU operations. Being able to retrieve a list of comments on a post will require (i guess here) a large proportion of MU's scripts to have been executed, hence voiding the cache's main benefit.
    I think it's ok to keep the cache refreshed with every comment, since most of pageviews are not resulting in a commenter, and therefore as you said in 80% of cases the cache will be used.
    I think the main issue here is to make sure that once the user changes something, the change is reflected on the site, or it will trigger a number of support questions of the type "i changed this and it doesn't work".
    Widgets are the main issue IMHO, getting them to refresh on the save page is already a good fix, but some widgets need more updates than that. Themes with specific admin pages will probably need a look too. All cached pages should be flushed everytime a theme admin setting is changed to reflect that change.

  13. quenting
    Member
    Posted 16 years ago #

    in fact, the best to handle widgets and other plugins (provided your version still has widgets as plugins) is to add the action hook to admin.php.
    I've added:
    if($_SERVER['REQUEST_METHOD']=='POST') {
    do_action('my_update_plugin');
    }
    to the plugins section of this file. This makes sure the hook is called anytime a plugin page is loaded with the POST method. This handles a refresh of cache for any change in plugins, which covers widgets, custome theme admin pages and your own set of custom plugins, provided changes are only made via form submission and POST method (a general good practice anyway).

  14. quenting
    Member
    Posted 16 years ago #

    Regarding the mclude/mfunc stuff, it's fairly easy to work with it.
    Basically, you need to output the required code as commented html in the page. When serving the page from the cache, the commented html will be replaced with a call to the function / an include of the file.
    This means that the function called/file included must be fairly stand-alone, or must include itself the necessary other files/functions to be able to run without wordpress being fully loaded. If it requires too much of wordpress, ucluding such file may negate the benefits of using the cache.
    I've personnally duplicated the necessary functions and added them to wp-cache-phase1 (so that they're visible without anythingloaded).
    Using this method it should be possible to replace the javascript SK2 functions.

  15. quenting
    Member
    Posted 16 years ago #

    here is the fix for the payload SK2 plugin to still work with WP-Cache. This also solves problems that may have occurred with the stopwatch plugin which uses the time embedded in the payload plugin (which as a side note, breaks the plugins independance, but well).
    This also gives a good example of how to use the mfunc thingy :

    In sk2/sk2_payload_plugin.php, replace the form_insert function with:

    function form_insert($post_ID)
    {
    $seed = $this->get_option_value('secret_seed');
    if (empty ($seed))
    {
    $seed = sk2_rand_str(10);
    $this->set_option_value('secret_seed', $seed);
    $this->log_msg(__("Resetting secret seed to: $seed."), 5);
    }
    $time = time();
    $ip = $_SERVER['HTTP_X_FORWARDED_FOR'];
    //echo ("<!--#". $time . "#". $seed . "#". $ip ."#". $post_ID . "#-->"); // debug
    $payload = md5($time . $seed . $ip . $post_ID);
    echo "<!--mfunc cached_sk2_javascript_payload('".$seed."','".$post_ID."') -->";
    echo "<input type=\"hidden\" id=\"sk2_time\" name=\"sk2_time\" value=\"$time\" />";
    echo "<input type=\"hidden\" id=\"sk2_ip\" name=\"sk2_ip\" value=\"$ip\" />";
    echo "<input type=\"hidden\" id=\"sk2_payload\" name=\"sk2_payload\" value=\"$payload\" />";
    echo "<!--/mfunc-->";
    }

    note the added mfunc tags. The seed and postids are passed as parameters so that the replacement function doesn't have to fetch them. WP-Cache is smart enough not to output those parameters in the HTML file received by the client, which makes you safe from seeing the seed hijacked.

    Now, add this function to wp-cache-phase1.php :

    function cached_sk2_javascript_payload($seed,$postid) {
    $time = time();
    $ip = $_SERVER['HTTP_X_FORWARDED_FOR'];
    $payload = md5($time . $seed . $ip . $postid);
    echo "<!--cached--><input type=\"hidden\" id=\"sk2_time\" name=\"sk2_time\" value=\"$time\" />";
    echo "<input type=\"hidden\" id=\"sk2_ip\" name=\"sk2_ip\" value=\"$ip\" />";
    echo "<input type=\"hidden\" id=\"sk2_payload\" name=\"sk2_payload\" value=\"$payload\" />";
    }

    this way, time and IP won't be cached and the stopwatch+payload plugins will still work. The javascript_plugin only computes some math stuff so it's not broken by WP-cache.

  16. quenting
    Member
    Posted 16 years ago #

    well, I've pushed my modified version to a first server. If anyone needs an incentive to using wp-cache, have a look at this CPU usage graph :
    http://www.creerunblog.fr/cpu-day.png
    and see the drop around 15:10.
    Any other performance tweaks I've ever done pales compare to this. And what you can see above is on a box that was fairly well tuned already.

  17. lunabyte
    Member
    Posted 16 years ago #

    That is a pretty big drop Q. Pretty decent indeed.

    I wonder how this would tie in with my ajax comment and paged comment plugins.

    Hmmm, guess I'll have to get back to playing with this as soon as I have the time.

  18. andrea_r
    Moderator
    Posted 16 years ago #

    Whoa.

    *favorites topic*

  19. quenting
    Member
    Posted 16 years ago #

    this plugin's effect is crazy.
    I have a very popular blog that's 1st in google for the name of a popular french TV show. Every day at the time of the end of the TV show, the blog receives huge amounts of traffic, with something like 3000 users online on this blog alone.
    This used to trash the server hosting this blog every day at that time. I needed to temporarily disable access to the blog for the whole server not to fall, otherwise none of the lbogs hosted there would be accessible for about 45 minutes.
    Well you know what, not today. Today i have 3000 users online on that blog, the throughput is 8MB/sec (without the images), and the load average is a g*dd*mn 0.47. CPU usage 36%. Like it's 6am lol.
    Man, have I had known this and exeprimented with it earlier, I might have needed half of my 8 servers. Maybe less.

  20. lunabyte
    Member
    Posted 16 years ago #

    Damn.

    What else can I say?

  21. suleiman
    Member
    Posted 16 years ago #

    that is honestly amazing.

    thanks a ton for the excellent work quenting. Time for me to jump on it!

  22. lunabyte
    Member
    Posted 16 years ago #

    If I'm ever in France again, beers and smash sandwiches for ya. ;)

    Mmmmmm...

  23. quenting
    Member
    Posted 16 years ago #

    actually i'm in switzerland, but i'll be happy to share a fondue if you stop by Geneva ;-).

  24. lunabyte
    Member
    Posted 16 years ago #

    I'll add it to my list. :)

    Someday, I guess.

  25. suleiman
    Member
    Posted 16 years ago #

    quenting, just to be sure you're dropping wp-cache into your mu-plugins dir right?

  26. quenting
    Member
    Posted 16 years ago #

    well, not really, i don't use the folder and dropped the various files directly in mu-plugins (i'm not sure for my current version but i remember my old one wasn't recursing into folders, so i took the habit of just dropping everything in mu-plugins. You need to do a bit of path switching here and there (basically replace plugins with mu-plugins).
    Also I don't use the lnk thing, i just copied the contents of phase1 to advanced-cache.php, because I have many synchronized MU installs and rsync doesn't like symbolic links much (at least my sync script isn't made to take them into account).

  27. suleiman
    Member
    Posted 16 years ago #

    okay thanks for clearing that up.

    Would you mind also sharing which code you modified to set it to default as enabled?

  28. quenting
    Member
    Posted 16 years ago #

    define('WP-CACHE',true);
    in wp-config.php
    and
    $cache_enabled = true;
    in wp-cache-config.php (put in wp-content with your settings inthere).

  29. suleiman
    Member
    Posted 16 years ago #

    thanks. I'll test it locally and see where it leads me.

  30. suleiman
    Member
    Posted 16 years ago #

    hmm..when i copy the contents of phase1 to advanced-cache.php I get the following error on my local install:

    Fatal error: Cannot redeclare wp_cache_postload() in /Applications/MAMP/htdocs/wp-content/mu-plugins/wp-cache-phase1.php on line 58.

    Obviously this is happenning because it's already been declared now in advanced-cache.php. Does anyone know of a way to skirt this issue?

About this Topic