Thursday, January 18, 2007

MySpace is Show Biz

Great article by David F. Carr on the behind the scenes mayhem at MySpace, whose exponential growth from a few hundred thousand users to 30 million users overloaded the system.

"MySpace started small, with two Web servers talking to a single database server. Originally, they were 2-processor Dell servers loaded with 4 gigabytes of memory"

Again and again MySpace was re-written to add: replicated databases, SANs, ASP.Net, and 64-bit servers. One interesting lesson was the addition of caching

To further lighten the burden on its storage systems when it reached 17 million accounts, in the spring of 2005 MySpace added a caching tier—a layer of servers placed between the Web servers and the database servers whose sole job was to capture copies of frequently accessed data objects in memory and serve them to the Web application without the need for a database lookup.

This sounds a lot like what GigaSpaces is offering. It's a LindaSpace-like type of middleware that replaces a messaging paradigm with a associative shared memory approach. Intense caching and heuristics to move data closer to its clients.

Scott Rosenberg makes the point that Web 2.0 successes may look easy from the outside, but behind the scenes there are huge challenges. Like Show Biz!

No comments: