HP breaks through: de-dupe is a core technology for the big guys

HP formally made a big announcement about de-duplication in a few VTL and HP products this past Monday. It was interesting to see the public reaction to this, or the lack thereof. I've heard some comments to date that there's nothing all that remarkable about the HP announcement, and it's just one more de-dupe appliance in the market. Simply not true. That announcement contained both a slightly less interesting external partner delivered de-duplication technology (for the VLS platform, and only because the external partner has had it for a while already) and the more interesting announcement that HP has developed some de-dupe technology internally (for the D2D2500 and D2D4000).

The apparently internal development of a de-dupe algorithm by a major vendor, even on an SMB/SME box, is a big thing. In my book, this means HP thinks of de-dupe as critical enough in the market place that they're willing to put real skin in the game (time and effort). Now they have core IP undoubtedly architected to support a broad range of potential future systems. Is this a differentiator compared to other big guys that are making acquisitions that may be less easily integrated into their portfolios? Time will tell.

It is obvious that de-dupe has rocked the world of secondary data storage over the past couple of years. But de-dupe is only getting started. Someday it will be a lot farther reaching than any of us can anticipate. As vendors start to gear up approaches to de-duplication, we're soon going to find that de-dupe rapidly becomes a pervasive technology in all kinds of products, well outside of the secondary storage and WAN optimization roles it plays today. HP's announcement this week suggests they see this. Kudos HP.

More to come soon on the subject of de-dupe.