Storage Clusters: they aren't coming; they're here

If you're retiring in the next five years you can skip this article. Otherwise, listen up.


RAID arrays were great in their day. But that day is drawing to a close. Managing LUNs and volumes, paying 20x the cost of the raw capacity for protection, poor scale-out: RAID arrays are just not competitive for large-scale infrastructures.




Storage clusters are now a proven commodity, with support from companies such as Oracle, IBM and NetApp. Highly resilient, simplified management, much lower cost. What's not to like?




Here are some examples:

• The world's largest data centers, Google, Amazon, Yahoo and Microsoft's MSN, all use storage clusters for 7x24 availability in their advertising operations
• At least a dozen more firms are selling cluster storage, including NetApp, the fastest growing large storage company
• Polyserve and Red Hat's GFS focus on storage clusters for Oracle and DB2 databases. With Oracle and IBM support.
• Omneon, a company specializing in storage and multi-media support for broadcasters, is selling "Media Grid" storage clusters. TV stations are a real-time 7x24 production environment: if the stuff doesn't work, the TV station doesn't get paid. It works.
• A storage cluster company - Isilon - just went public with a $1.4 Billion market cap

Arrays aren't going away tomorrow, or ever. It took 10 years from the publication of the Berkeley RAID paper before RAID arrays took 50% of the external storage market. Yet storage cluster use has been growing rapidly in some major niches: internet data centers, video and broadcasting and web services. The 85% of enterprise data that is unstructured is the next big market for storage clusters.




Even if you work for an IBM-only shop you should know that IBM Global Services is installing and supporting storage clusters today. If your CIO is more aggressive, start getting informed about this technology today. You'll be the more valuable for it.




Some brief intros to storage cluster tech from my site StorageMojo include Google File System, Microsoft's Boxwood, Google's BigTable storage system, and Isilon's Cluster Technology. Google's stuff isn't for sale, nor is it architected for enterprise use, but they've done a good job of distilling storage clusters down to their bare essentials and exposing the issues.




I also recommend Kevin Closson's Oracle blog for deep Oracle insight.




In five years you could be managing petabytes with fewer headaches than terabytes give you today. Storage clusters are a new day for enterprise storage and data management pros.




Comments welcome, of course.