Defining the cloudy cloud: the next gen infrastructure?

Just recently, I published an article in print over at InfoStor about Cloud Based Storage, a term I used to refer to user accessible, file-based storage in the cloud (much like Amazon S3 or Nirvanix, or a number of other solutions soon to come). Dave Simpson in his editorial comments made a reference to vague terminology around the cloud and hosted services, but frankly I think we're on a path to improvement. Here's how I see cloud terminology playing out.

When we talk about cloud computing or storage, we're fundamentally talking about a whole set of infrastructure components and services that create a remote environment that can be used for various purposes. The application layer on top of this stack makes up SaaS or hosted services. I see the cloud as being a parallel to traditional infrastructure, but with a whole new set of requirements and capabilities, e.g. user self service, elastic in performance and capacity, etc. (wait for the article). This "cloud infrastructure" is really just a superset of a traditional infrastructure when it comes to capabilities, but perhaps a subset of a traditional infrastructure when it comes to systems. And with this view "a cloud" might be used within an enterprise's own network to create an internal service provider model, or served up by an external service provider. Moreover, an external service provider's cloud infrastructure might be used by other application vendors to host their products for resale as a service by VARs or other vendors or service providers.

Add to the cloud bit that it can enable new service models in the enterprise with tremendous TCO benefit, and for the end user, it will enable a rich new range of services. But it takes a unique new approach to architecture. Personally, I believe the vendors most able to bring solutions to bear in some layers of this space will be software-centric solution vendors, the likes of IBRIX, FalconStor, etc. Hardware will absolutely have a place, but modular, lightweight software services may make or break the capabilities of this infrastructure when it comes to the unique services that cloud infrastructures will require.

So does Microsoft's Azure have it (and maybe the bigger question is, do they have enough openness and extensibility)? Does Amazon have it? These infrastructures are most about development mind share and development community today, but evolving rapidly into sophisticated distributed, scalable web service clouds. A bunch of other folks are building cloud infrastructures behind application hosting environments with similar ambitions for extensibility and scalability - SAP's relatively stealth ByDesign might be another. I see similarly or more capable emerging cloud solution sets on a wide number of fronts that could allow even middle tier, regional service providers, or even vertical niche providers (maybe Yahoo's Zimbra for Universities) to build killer cloud infrastructures with vastly superior capabilities. If these folks are fleet on their feet, they may even create cooperative efforts where services could be replicated, moved or managed across different providers with transparency, and truly challenge the big guys. Undoubtedly, standardization and portability will be the name of the game. But a factor in the regional service provider's favor, they can't roll their own unique architecture, and may in fact choose off the shelf solutions that are the same solutions enterprises are using to create internal clouds.

Whether we see these solutions offered up as full cloud solutions competitive with MS Azure or Amazon S3 may be an unanswerable question, but these cloud infrastructures will be there. The regional service provider depends upon it. To differentiate themselves in the eyes of the user, and provide richer, revenue enhancing services, the service provider must think about cloud infrastructures to host their own flexible, adaptable, extensible solutions. To me, it only makes sense they'll serve up the full capabilities of these infrastructures to end users, but time will tell.

The thing is, I think most of the industry is just starting to figure out the cloud today, with the exception of a few thought leaders. If you look back through the history of the Internet, whenever we've moved a service or capability outside of the traditional data center model, we've received new capabilities in return. This time around, we're charging full bore ahead with the infrastructure layer (trust me, there is more to come). But you should be asking yourself when you look at the cloud, what do you get in return? Scalability and low cost may not be enough. But as the next generation of solutions start popping out of the box, they will be offering you more, in the way of management, control, security, data protection and backup, visibility, automated reconfiguration, and eventually, even the possibility of better availability than you may have today. Cloud-based storage is the first out of the box solution set to charge forward into this space with a new set of capabilities that we didn't have before - it exemplifies how we need to be thinking about Cloud technology. Similar next generation capabilities are what will make the Cloud a compelling opportunity.