Cloudy clouds and standards

I had an interesting conversation about standards in the cloud just the other day. We're seeing a lot of buzz about standardization, but if you're out in the trenches using Amazon EC2 today, SalesForce, or an assortment of other cloud technologies, does the cloud really look to you like anything you can imagine a set of standards for? Frankly, it often seems that on one hand, the cloud is way past the point where we can talk about standardization, and, on the other hand, it seems that the cloud is so fuzzy and ill-defined that we can hardly approach standardization in any useful way. But the thing is, this conversation often rapidly spirals out of control because our perspectives about standardization are historically induced to be pretty binary - we look at standardization as an all or nothing approach; either we have a comprehensive, far reaching set of APIs that can do almost anything cloud related (even taking SOAP to the next level), or we have nothing and this motivates us to be free form in our innovation and thinking.

My insistence is that a Cloud API needs to be neither of these. It is not an all or nothing choice. For this market to thrive, especially in light of current dominance by some very big players (Amazon, Google), secondary and emerging players need better interoperability to lend them collective competitive strength.

For the hosting provider, ISP, or any other service provider smaller than the Amazon's of the world, they must have some standardization that ensures their customers that they can turn to the cloud in a long term way, without provider lock-in. Moreover, interoperability is the only way that customers can easily hope to incorporate multiple providers to reduce shorter term sticky issues around availability.

Without going down the path of RESTful APIs and how that will in fact create some level of standardization, when it comes to defining standardization, in my opinion, there only needs to be standardization around a few core "activities" that are targeted more at interoperability than uniform services and structure. The naysayers have a good claim that standardization could stifle innovation, but what I care about, as an end user, is really carrying out a couple of key steps, in the same way, regardless of who the provider is.

But users need to expect when they come "up" the cloud stack, they're going to buy into more lock-in. For example, I don't think anyone on the face of the earth thinks there's hope for standardizing much of anything between something like SalesForce and SAP byDesign. At the same time, those users realize that those solutions are pretty extensible, and that they will eventually be able to store, move, and protect their data in any way they want.

The interesting thing, though, is that we've been talking about and evolving this self describing capability in SOAs for a long time now, and I think we are on the cusp of a "discoverable" solution ecosystem in the cloud. Conceptually (meaning currently out of touch with reality, but eventually plausible) you could have a really complex application where you are using proprietary APIs, but be able to kind of crawl a web of potential services and see which APIs could be replaced or duplicated by other services. Conceptually again, this could mean distributing your application and data across many different providers. Conceptually again, this might mean someday we have some big search engines and/or applications that can examine an application or something you're doing on the cloud, and figure out how to move it or distribute it for better availability. Interesting stuff to think about. Back to reality, the reason I'm talking about this conceptual stuff is that standardized APIs aren't clear at all. We're really talking about a real web of services, where different providers might have different variations of services and only carry out some common functions between them. For example, this one might provide some core database service, and this one might provide ETL, this one provides some data-centric service we haven't even thought of today, and this one might provide all three, but on someone else's block storage that doesn't have a file interface, etc. Standardization won't encompass the entire cloud ecosystem, but is going to come down to very basic core actions, and then we're going to have some defacto standardization in other areas where competing vendors want to offer plug-n-play replacements for the leader.

Perhaps the biggest thing, though, is that I hear the industry talking consistently about a Cloud API. Overall, I can't stand that term. There cannot ever be a Cloud API in a standardization sense. Cloud APIs will surround us, and that won't be the layer that is ever standardized. An API is just an abstraction of actions you're carrying out. A data and services interchange layer is what needs to be standardized. Moreover, if we do nothing, some defacto standardization is going to happen. The cloud is about easy extensibility. It is easier than ever to connect things, and translate between things. Writing a gateway between RESTful services will likely be a piece of cake, so if we never standardize as an industry around specific actions related to data movement and portability, then third parties certainly will. For my part, I even expect that to encompass pretty complex non-cloud APIs -- I'd bet money that within a year we start seeing people pop up with XAM, or Centera, or RISS, etc. to cloud storage gateways and a whole slew of similar services.

At the end of the day, what might come from cloud standardization is moderation of our expectations around huge standards efforts, and more effective interoperability in the data center. Let's call it cloud-induced openmindedness about exposing services and functions. If you're a vendor, and you aren't in this mindset, I can almost guarantee you that the cloud will shift the way you interoperate and expose services, whether you like it or not. When your customers start experiencing the extensibility of the cloud, you'll eventually find your customers turning to the cloud first if they can't find similar easy extensibility in your products.