The MU forums have moved to WordPress.org

Infrastructure for Scaling to Multi-Million Users and Hundred Million+ Blogs (17 posts)

  1. amirbq
    Member
    Posted 16 years ago #

    Hello,

    What sort of scale has people been able to achieve with WPMU?

    What type of Infrastructure Design would be required for having many Hundreds of Thousands to Millions of Users in a Blogging environment actively participating in a WPMU environment?

    Here are some elements to consider, but not limited to:

    1. Number of Servers (CPUs, RAM, NICs, etc.)
    2. Storage Internal/RAID or SAN, etc.
    3. Network Size / Bandwidth
    4. High Availability??
    5. Database Scalability / Clustering
    6. Serving: Text / Audio / Video / etc.
    7. Other

    Any input would be greatly appreciated.

    Thanks.

    Amir

  2. rbytes
    Member
    Posted 16 years ago #

    We use
    1) Core 2 Duo E6600 (1066MHz FSB / 4M Cache), 2GB DDR2 RAM
    2) 2x160GB SATA2 HDD
    3) 1tb Bandwidth at 100mbit switch
    4) 100% :)
    5) single server

    good luck

  3. dsilverman
    Member
    Posted 16 years ago #

    If you're planning a deployment of that size you probably need to find at least one experienced systems administrator and have a good amount of money to spend. You'd want to test various configurations and setups and see what works best for you at the expected load. You'll want to look at the multi-database functionality in the code but also be prepared to do a lot of your own work customizing things for your environment. You'll need web servers, database servers, load balancers, switches, shared storage, etc. You'll need fat pipes, cooling, power, etc., which probably means a cage or a rack in a data center somewhere. If you're planning something of this scale, you're going to get a lot better advice by hiring someone who knows what they're doing than the non-specific recommendations of random folks on a message board who do not have deployments nearly that large.

    I won't tell you my specs because they're irrelevant.

  4. andrewbillits
    Member
    Posted 16 years ago #

    "Hundred Million+"? It's definetely possible but not out of the box. As for the number of servers, "dozens" would be an understatement.

    Let me know if you'd like to discuss this further.

    Thanks,
    Andrew

  5. andrea_r
    Moderator
    Posted 16 years ago #

    "What sort of scale has people been able to achieve with WPMU?"

    Does wordpress.COM ring a bell?

    I can't believe no one has pointed that out yet.

  6. lunabyte
    Member
    Posted 16 years ago #

    I thought that was a given?

  7. andrewbillits
    Member
    Posted 16 years ago #

    Same here.

  8. drmike
    Member
    Posted 16 years ago #

    I'm writing it into a fanfiction as the 518th layer of hell. Does that count? ;)

  9. lunabyte
    Member
    Posted 16 years ago #

    I think it does, but if it's like any other documentation, it won't get read except by the ones who really didn't need to in the first place.

  10. andrea_r
    Moderator
    Posted 16 years ago #

    God, I love you guys.

  11. mysorehead
    Member
    Posted 16 years ago #

    Google found Barry who says this about wordpress' setup http://barry.wordpress.com/2007/04/16/additional-capacity/

    and this from last October

    * 80 physical processors
    * 139GB of RAM
    * 91 hard drives with a combined total of over 15 terabytes of storage space
    * 2000 database queries per second spread over 40 MySQL instances
    * Over 8 million objects stored in memcached serving over 8000 requests per second

    Richard

    PS> Barry also has other interesting things to say about load balancing, caching, .....

  12. mysorehead
    Member
    Posted 16 years ago #

    Matt and Barry's slideshow from wordcamp on high performing sites can be found at:

    http://barry.wordpress.com/2007/07/22/high-performance-wordpress/

  13. drmiketemp
    Member
    Posted 16 years ago #

    I'd have to do digging for the link but when they were shopping for their third datacenter, they released teh specs for what they were locking for. They really weren't that high powered computers. Like half a gig of memory on each, etc.

  14. mysorehead
    Member
    Posted 16 years ago #

    That post was on Barry's blog as well.

  15. drmiketemp
    Member
    Posted 16 years ago #

    Locked down terminal. Can't go digging for it. Sounds right though. barry seems big on the hardware side of the wp.com site.

  16. honewatson
    Member
    Posted 16 years ago #

    Memcached seems pretty awesome for large scale deployments:

    Facebook has roughly 200 dedicated memcached servers in its production environment, plus a small number of others for development and so on. A few of those 200 are hot spares. They are all 16GB 4-core AMD64 boxes

    http://www.skrenta.com/2007/05/scaling_facebook_hi5_with_memc.html

  17. honewatson
    Member
    Posted 16 years ago #

    Not that facebook uses wordpress but it is php.

About this Topic

  • Started 16 years ago by amirbq
  • Latest reply from honewatson