Web 2.0 Data Scalability

Image via Wikipedia In this previous post I wrote a section on Web 2.0 data platform scalability, in it I mentioned the importance of scale out. I also mentioned that scale out was important but currently it is hard. There…

Image via Wikipedia

In this previous post I wrote a section on Web 2.0 data platform scalability, in it I mentioned the importance of scale out. I also mentioned that scale out was important but currently it is hard. There really aren’t a lot of “elegant” or “natural” ways to scale out a database if you are using any of the mainstream database vendor products (SQL Server, MySQL, Oracle has grids but isn’t shared nothing scale out so is limited).

A question I was asked was “Surely this has all been sorted by now through their widespread use in the enterprise?”. Good question but unfortunately the answer is no. The reason behind this is simple. There are very few enterprise workloads that cannot be satisfied with a scale up (big server) solution. Maybe 1 in every 20,000 databases will have a workload so great that it can’t be satisfied by a single server, in fact most enterprise databases will happily share servers with other databases. The biggest ERP’s and online share trading applications that I am aware of all are provided for by a single big server solution. Quite simply, none of these applications have the workload requirements of tens of thousands of concurrent users that a Twitter or Facebook has.

So this issue really hasn’t been addressed in any adequate way by the mainstream vendors. Later on I will post on why the current products are not suited for scale out, and the efforts in progress to rectify this.