The MU forums have moved to WordPress.org

Amazon S3+WordpressMU (27 posts)

  1. bigdoug
    Member
    Posted 14 years ago #

    I've been asking a lot of scalability questions recently, I know...really concerned about hosting podcasts, videos, etc and want to control costs in a scalable way.

    I've been thinking about Amazon S3 today -- some advantages over traditional storage, including that you get a built in CDN.

    Turns out there is a plug-in for plain old WP that maps uploads onto Amazon's S3 service (over at http://tantannoodles.com/toolkit/wordpress-s3/).

    Has anyone experimented with integrating S3 with their MU site? Any ideas on how close TanTan's plugin gets to MU compatibility?

  2. honewatson
    Member
    Posted 14 years ago #

    This plugin is really great. Not sure if it works with Mu but it can be set up to run off a subdomain of your main domain.

    As long as you can protect the user info from the users this would be a great plugin.

  3. sheatsb
    Member
    Posted 14 years ago #

    it works with mu. you just have to fill out the config.php file with the default details. i had a hack that allowed me to put it in the mu-folder.

  4. andrewbillits
    Member
    Posted 14 years ago #

    S3 bandwidth isn't cheap so make sure it's the most cost effective option for your site.

    Thanks,
    Andrew

  5. quenting
    Member
    Posted 14 years ago #

    s3 is used by wp.com.
    To use it effectively, it's best to have an in-between cache server from which you'll serve static files most of the time. You may use apache+mod_cache, squid, or stuff like varnish to do this (i think this last bit is what's used on wp.com).

  6. honewatson
    Member
    Posted 14 years ago #

    I'm interested to know who has cheaper bandwidth than s3 andrew?

    Nginx is excellent for serving static files also.

  7. sheatsb
    Member
    Posted 14 years ago #

    The only service with cheaper bandwidth I know of is Dreamhost, but they're not terribly reliable. Also, Amazon is not a true CDN (it doesn't find the server closest to you for content delivery), but there's this new service with a somewhat promising API called Nirvanix that is a CDN.

  8. quenting
    Member
    Posted 14 years ago #

    nginx doesn't do caching afaik.

  9. andrewbillits
    Member
    Posted 14 years ago #

    Nope, nginx doesn't do caching.

    Thanks,
    Andrew

  10. honewatson
    Member
    Posted 14 years ago #

    Nginx doesn't do caching out of the box but a lot of people have found it performs as well as squid or apache+mod_cache when it comes to serving static files.

    There is a new caching mod that has been released for nginx.

    http://www.ruby-forum.com/topic/139341

    sheatsb - dreamhost is can be unreliable at higher bandwidth usage. Nirvanix looks interesting. Thanks for the tip.

  11. sheatsb
    Member
    Posted 14 years ago #

    i also have the hack that makes the plugin work in the mu folder, if anyone one wants.

  12. andrewbillits
    Member
    Posted 14 years ago #

    Feel free to upload it to wpmudev.

    Thanks,
    Andrew

  13. quenting
    Member
    Posted 14 years ago #

    honewatson, nginx is indeed great if you have an in-house solution for serving static files, but in correlation with amazon S3, you really want to do caching because you're charged by bandwidth usage.

  14. honewatson
    Member
    Posted 14 years ago #

    What would be quite interesting is combining s3 with supercache and nginx.

    If you could some how transfer the html of the supercache to s3, nginx can easily be used to proxy files from s3.

  15. podcastnyc
    Member
    Posted 14 years ago #

    Does anyone have the info on the MU folder hack for the Tan Tan Noodles s3 plugin? I've been trying it out and would like to set it up outside the view of my users. Thanks in advance.

  16. podcastnyc
    Member
    Posted 14 years ago #

    Check that. I got the plugin working by using the plugin that allows plugins with subdirectories in the MU folder.

    http://heftagaub.wordpress.com/2007/04/06/supporting-subdirectory-plugins-in-wordpressmu/

  17. zappoman
    Member
    Posted 14 years ago #

    Glad to see someone's using my plugin.

    By the way, I'm not yet using S3 for file uploads (it's on my list)... but I did implement a solution for off-loading all of my theme resources to S3. I wrote a small plugin to do this (not much point in sharing the code because its so simple and assumes that you'll structure your S3 buckets according to a specific mechanism)...

    The prinicipal is this...

    Hook the stylesheet_directory_uri filter...

    add_filter('stylesheet_directory_uri' , 'YOUR_FUNCTION',1);

    YOUR_FUNCTION will need to take the normal mu-style URL for the theme path (usually: /wp-content/themes/theme_name/) and map it to your S3 bucket.

    Anyway, last month we served about 1 million "hits" from S3 and got a bill from Amazon for $1.54. Not to mention the fact that we all that load off of our server.

    Pretty reasonable.

  18. podcastnyc
    Member
    Posted 14 years ago #

    It's a good plugin. Thanks for the plugin and the tip on off-loading the CSS as well.

  19. zappoman
    Member
    Posted 14 years ago #

    One more thing... for my S3 bucket scheme, I use...

    themes.domain.com/theme_name/version/...

    I do this so that I can set far future Expires on the theme components. If I need to change a theme, then I change the version. I do this be reading the theme info out of the theme (style.css) inside of my stylesheet_directory_uri filter, and building my theme URL from that.

  20. selad
    Member
    Posted 14 years ago #

    We use a very similar approach. Our simple plugin for off-loading themes css and image files can be found here.

  21. PerS
    Member
    Posted 13 years ago #

    Hi, If anyone are using this i.e. off-loading blogs.dir to S3, could you please share your experience ? :)

  22. honewatson
    Member
    Posted 13 years ago #

    Couple of points:

    1. Somebody complained about s3 bandwidth costs yet nobody has suggested a cheaper alternative. I'm interested if someone knows of one which allows you to pay on a usage basis.

    2. Nginx has a function called proxy store which allows you to store regularly proxied files from s3 and serve them from your server.

    3. A number of people have found that Nginx can outperform projects like Varnish when it comes to static files.

    4. For core wordpress css, js, amazon cloudfront would probably be a better option than s3. Considerations need to be made for gzip. Cloudfront would also be interesting for supercached files.

    5. Lunabyte has mentioned you should not be solely reliant on s3 or other amazon services and I agree.

    6. Puts to s3 don't seem so fast.

  23. PerS
    Member
    Posted 13 years ago #

    Thank you,

    I believe I'll start by uploading the templates to S3

  24. sunethj
    Member
    Posted 13 years ago #

    I think it is better to continue this conversation with a problem I face. I'll be hosting multiple domains with wpmu using multi-site manager plugin.

    Will I be able to use this plugin http://tantannoodles.com/toolkit/wordpress-s3/ to allow all blog users to store their images in one amazon s3 account (I can arrange them to use unique naming for their files)?

  25. gpo1
    Member
    Posted 13 years ago #

    am interested in this

  26. honewatson
    Member
    Posted 13 years ago #

    @sunethj

    I think wordpress.com stores all their images on wordpress.com sub-domains regardless of the domain.

    Amazon Elastic Block Storage and especially Amazon Cloud Front are probably better long term solutions than s3.

  27. gpo1
    Member
    Posted 13 years ago #

    can be adapted to make users upload too amazon s3 or next service above for media

About this Topic