The network virtualization frontier - virtualizing with Incipient NSP

As I mentioned just over a week ago, we've been beta evaluating a SAN-based storage virtualization platform (Incipient's NSP - www.incipient.com) for use here at the City.  I mentioned at that time that I'm impressed with the product and eager to see it released to the marketplace.  At that same time, I mentioned I'd write a bit more detail about feature-sets and capabilities as well as the challenges I see for that type of product.

The killer feature that the NSP product proposes to bring to market is enterprise-level storage virtualization in the fabric.  This is a bit different from any of the virtualization products out there.  Much of what's available as in-band I might classify as mid-range rather than enterprise.  What's available as enterprise is generally either out of band or isn't in the fabric.  As I've proposed before, it has always seemed to me that the intelligence for my SAN needs to be in my fabric.  It just makes sense -- I want to work with the same toolset regardless of what my storage is, and storage changes too fast to be constantly swapping that toolset out.  Fabric sticks around a while, and feature-sets tend to remain more highly compatible across revisions.  So remove some performance limitations by distributing processing and management, and it seems like fabric-based storage management may stick around for a long time.

The Incipient product is interesting because they're chasing enterprise-level scalability with a highly distributed architecture (storage switch port processing).  We've been reviewing clustering of a couple of Incipient NSP nodes, but I don't see any roadblocks on their map to multiple node clustering.  That will be the first product I've seen in the fabric really capable of enterprise scalability with minimal limitations and overhead.  On top of this scalability, they've built a nice product set with the core features you'd expect in enterprise storage technology (basic storage classification for management and delegation, copy, snapshot, etc.), plus some extra capabilities which are what I really want out of fabric-based tools, such as network-based volume concatenation and striping.  Their value proposition is strengthened because they're building on top of an impressively scalable architecture in Cisco MDS fabric switches.  In reality, they represent almost a third generation of development on this platform, with IBM Storage Virtualization Controller (limited scalability) and Veritas Storage Foundations for Networks (pulled from the market) having already come to market, albeit in limited fashion.  Incipient, compared to the others, has set the stage for what an enterprise toolset in this area should look like.

From what I've seen, anyone dealing with multi-vendor storage management should be paying attention to Incipient.  This might be you if you're looking at storage migration between vendors, storage migration with support for multiple vendors afterwards, storage vendor multiplication, or you just need to establish fixed management practices for your SAN over a longer period than storage growth and replacement will allow.  If most of us are looking at 30% to 40% annual dataset growth, chances are your disk is likely to cycle or expand significantly in 3 years. What are the chances you're going to be able to buy the same stuff in three years?.

But while you're paying attention, I think there are a couple of very important things to pay attention to.

First, Incipient has a battle on their hands to emerge as a front-runner amidst the competition from traditional SAN vendors. 

There's a lot at stake, and there are some pretty big players in this game.  The innovative ones might get the big picture and understand the value that a tool like this can provide to enterprise.   Consequently, this could make some of the big player's enterprise frameworks, i.e. Openview, Tivoli, etc., more attractive.  The innovative ones might pick up a tool like this, and then the possibilities for further integration might be endless.  The traditionalists likely won't step away from controller-based storage and the marketplace presence that comes with some of this today, even though failure to step away may result in not having that marketplace presence tomorrow.  And then you have the grey middleground where traditional controller-based disk vendors are extending their controller capabilities by breaking them away from disk (i.e. HDS Tagmastore).

There's plenty of muscle for the incumbents to squash the innovator here.  That has certainly happened often enough in the tech industry, but I also see a hugely cooperative effort being executed by the innovator in this area, and my money is on something impressive coming together this time around.

Second, Incipient is uniquely positioned to bring some incredible standard functionality into the enterprise.  They will need that functionality to be prevalent enough to drive implementation of unique features and to allow them to penetrate the enterprise market better than smaller niche players with unique featuresets.

Recognizing that none of this is based on anything from Incipient in any way, take a walk down imagination lane and think with me about what a completely mature fabric-based storage management system like this could eventually provide. 

Once you're doing the fairly complex basics of shuffling storage extents, volume management, and SAN copying, that establishes a foundation for most of the functionality on the marketplace.  Who are some of the more interesting front runners in storage right now?  3PAR?  What is 3PAR doing that couldn't be duplicated in a sophisticated network-based storage infrastructure?  Thin provisioning?  Seems easy.  What is Compellent's value proposition?  Nice hardware employing some innovative capabilities for managing the location and optimization of data in the background. A network-based toolset seems even better positioned to reshuffle data blocks on storage and reallocate extents on appropriate performance or cost disk, and then you can pick your own disk.  Dynamic growth?  Integration with SANTap?  In fact, maybe it's easier to think about storage features a network-based toolset couldn't provide given enough time.  But I want some of these capabilities today, and as I move toward a likely implementation of Incipient, I'm wondering just how much time that might be, if ever.  For us, the features of Incipient far outweigh these other capabilities, especially when the cost advantages of having a wide range of storage choices are factored in, but there's certainly a value proposition for those other features which is not easy to walk away from.

So, in imagining where tools of the future might go, the possibilities seem to be pretty broad given the right foundation.  Incipient seems to have a great foundation. The struggle will be navigating the waters that the big guys play in while building an enterprise market, and growing enough truly innovative features to build broad market appeal, and consequently surviving amidst a wide range of smaller storage vendors cranking out those same innovations.

For those of you with more interest, Richard Bourdeau, the Product Manager over at Incipient, has a great objective article at Storage Networking World discussing some of the issues surrounding fabric intelligence (http://www.snwonline.com/behind/sans_04-10-06.asp?article_id=680), and the news page at Incipient's site has a good collection of diverse articles surrounding fabric intelligence and the Incipient NSP product (http://www.incipient.com/news/news.htm).