Industry


Ads by TechWords

See your link here


Robin Harris's picture
Robin Harris

Random Writes

ILM: rest in peace

One of the most misguided computer industry marketing initiatives ever is coming to a slow and shuddering halt. People just don't have time for nonsense. Sadly, Information Lifecycle Management (ILM) is particularly seductive nonsense. It has taken far too long for practitioners to see past the vendor's honeyed words.

As I noted over at StorageMojo.com, the good folks at StorageNetworking.org have raised a red flag over ILM in three reports: Information Lifecycle Management in Perspective: Initial Findings From Surveys of Top Management , Information Lifecycle Management: An Analysis of End User Perspectives and ILM Survey: What Storage, IT and Records Managers Say. Each is the product of their own independent research.

The results are, IMHO, pretty damning, especially three years into a major industry marketing program. In short (and I highly recommend looking at the research yourself if you are considering an ILM project or are in the middle of one): there is no definition of ILM; people believe ILM can create more problems than it solves; and a general concern that technology is being thrown at a soft problem.

The Bastard Child of Opportunism and Necessity
Born in the still-smoking rubble of the dot-bomb aftermath, mid-wifed by EMC's powerhouse marketing, ILM was intended to return EMC to the halcyon days of Y2K, when F1000 glass houses were buying TimeFinder and Syms like popcorn. In 2001 EMC had forecast FY03 sales of $12 billion and instead booked about $6.2 billion, one of the most startling industry u-turns since Osborne Computer imploded. Battered and bloody, a drowning EMC sent forth its sales legions and camp followers to spread the good news: ILM is here and will save your bacon. They actually meant "our bacon".

ILM = I Like Money
It is an important function of marketing to help practitioners create a story that gives direction and meaning to their work. Most IT folks live in an interrupt-driven world and don't have near enough time to step back and figure out the strategic initiatives. Good marketers listen hard to their customers, look at the technology and business trends affecting all businesses, and find the sweet spots where it is a win-win for customer and vendor alike. ILM failed to meet that standard. By embracing it, EMC in particular let the entire storage industry down. Oh, and thousands of customers. So when you hear "ILM" think "I Like Money". That is what it is all about.

Why ILM Failed
Some note that ILM is simply Hierarchical Storage Management (HSM) with a new coat of paint. HSM has never taken off outside some narrow niches because it has never been able to compete with the rapidly dropping price of storage. Most folks would rather spend $250,000 to solve today's problem rather than tomorrow's. And today's problem is usually "we're running out of capacity and IOPS."

Others point to technology problems for ILM's demise, which is valid: products to auto-magically classify and migrate data are still not here three-plus years later. Yet if that were the only problem there would be hope.

In my view, the largest problem is organizational: ILM assumes that IT "owns" the data when, in fact, IT is the custodian. So we get hand-waving about "working with business units, the end-users" and so on. Sounds good, until you try it. The business units don't want to work with IT to classify their data. They don't care about IT's storage problems. They want to keep all the data they might need and they want it whenever they want it.

The thought balloon over the business-unit manager's head is "my people classify the data and IT saves money I'll never see." Uh-huh. We'll get right on that. Next!

There are niche cases where ILM may be feasible and economic. Project-based organizations, where the projects have completion and closure, such as direct mail houses, media production, research projects and the like, may benefit by shuffling completed projects off to a big JBOD and an archive. The data is classified by the project it is associated with, while there is limited need to access it once the project is "in the can".

ILM Makes Sense to Vendors
For vendors, ILM is a wonderful strategy.

That's because the only reason to have tiered storage, HSM or ILM is because the primary storage is too costly to use for everything. As Jim Gray, another all-around really smart guy at Microsoft Research said in a must-read interview “The two things that are going to be real shifts in storage are tertiary storage going online so there is no distinction; and intelligent storage, so that we raise the level above SCSI.” What Jim is saying is now that disks are as cheap as tape, the role of tape libraries will be absorbed by disks. So the storage taxonomy will be: fast disks for databases; slow cheap disks for everything else. What does ILM mean in a two tier, application driven infrastructure? Nothing, that's what.

So the more practitioners spend on ILM, the more they enable vendors to keep selling complex and expensive storage. And since it is unlikely that the ILM project will meet the its unrealistic goals, it's a loss for the customer both ways.

ILM As A Marketecture
Even Dave Hitz, a co-founder of NetApp, wrote in his blog ILM is about everything; therefore ILM is about nothing. Dave identifies several areas of confusion, the biggest ". . . is that EMC took the idea of ILM and decided to use it to explain everything that they do. At EMC, everything is about ILM." I agree with Dave. And as such, it is the largest single storage marketing misfire since Ellen Hancock kept IBM from building RAID systems in the mid-1990's.

The Array Industry vs. The Storage Industry
Business history is full of instances where entrenched companies dismissed competing technologies: Western Union refusing to buy Bell Telephone; railroads ignoring trucking; 8-inch disk vendors not investing in 5.25" disks. The array industry has forgotten it is in the business of providing cost-effective data storage. That may not always mean storage arrays.

The RAID concept galvanized the industry, just as 16-bit minicomputers once did. Yet just as the need for mini-computers passed with the rise of microprocessors, standard shared-cache storage arrays will not be the preferred model for massive storage in ten years. They aren't at Google and Amazon. If your enterprise IT has to compete with them - and if it isn't today, wait a year - you won't want to try from a high-cost infrastructure.

So in the failure of ILM there is an opportunity to get the industry to start focusing on what customers really want: storage that provides cost-effective, reliable performance and availability. Not excuses to buy over-priced and over-hyped products that overshoot most customer requirements.

What People Are Saying

The reason for the failure

The reason for the failure of ILM is that it takes into account only the procurement cost of the various tiers of storage. That the author states the problem is only capacity and IOPs plays into the alleged value proposition of ILM.

It's an unfortunate fact that the majority of the real cost of storage goes into management. ILM may address to the capacity and IOPs problems to a certain extent, but it introduces vast new management costs, as the problems of backup/recovery, disaster recovery are not addressed for the most part. When implemented the EMC way, it introduces new capital costs as you add the tiers 2, 3 and 4 (clariion, centera, and VTL). Each of these add complexity to the storage and network, so it multiplies the cost of the total solution - especially where it hurts the most (data management).

More importantly, there's no efficient automated way to intelligently manage the relationships between the tiers. There may be one solution for your users' home directories, another for your structured email data, and another for your databases. You have to deliver another solution for your regulatory compliance data as well.

So you end up having to take up ILM initiatives for all of your different types of data. And each one of them will have to get their own backup and recovery schemes, as well as disaster recovery schemes.

So ILM may make sense, but only if it's a unified approach. This is where EMC's marketing went beyond what their technology could deliver.

Right On! LeRoy… I agree

Right On! LeRoy… I agree fully

Let”s Start Here with one ILM Strategy…
If you see value in ILM then your IT shop has a need for it.
Shops, and people, who do not see value in ILM have no need for it.

For years I designed and Implemented what I called “Rolling down and up the Storage Hill”. A lot of people thought I was being humorous?
One day someone asked me if I had ever heard of ILM.
Turns out to be a real nice sounding name for “Rolling down and up the Storage Hill”. I never looked back.
Some people would replace “down and up” with the better sounding “up and down’. I don’t because more Information goes down the hill than ever comes up in classic IT. There are some exceptions in the Web area.

Vendors are commission sales agents so they try to sell into the ignorance, not stupidity, of the clients who trust them.
ILM is not for everyone and even some of the people who need it do not have the infrastructure to support it nor the money to acquire that infrastructure.

As a rough rule of thumb you need an IT infrastructure with the bandwidth capability of 1 TB per hour minimum. This is from every possible source to every possible destination.

This is why it is necessary to know, or find out, the “Speed Limit of the Information universe” for your IT shop.
An I/O map is also very handy, if not essential.

Fortunately, not all of your Information needs ILM. Like HR Information, unless you are an HR Outsourcing firm. Then it is your life-blood and you need ILM if your stored Information exceeds the Storage capacity of your laptop.

The 5 Phase ILM cycle is:
<== Assess ==>
<== Strategy ==>
<== Design ==>
<== Implement ==>
<== Manage ==>

The "<==" and "==>" symbols indicate a circular process.

The IoD (Information on Demand) concept is the foundation on which ILM, and other processes, is laid.
My definition of IoD is:
IoD is a concept to achieve a Strategic goal. IoD requires the Pervasive Information Fabric using agents and multi-agents. To achieve this architecture the ILM (Information Lifecycle Management) phases of Strategy, Design and Implement must coordinate. The ILM phases of Assess and Manage are monitoring activities of the Pervasive Information Fabric.

E2EIoD is “End-to-End Information on Demand”.
I actually spent years working in obscurity developing this only to find IBM had done a better job and called it SOA.

Some of the pieces you will need are:
E2EIoD Information Areas and Processes
Information areas are:
1) Online
2) Nearline
3) Offline
Basic IT / E2EIoD / ILM Processes are:
1) Configuration
2) Replication
3) Migration
4) Re-Configuration
5) Redeployment

That’s all there is to it!

Thoughts for the day?

So what's in a name?
Is it E2EIoD or SOA?
Rolling down and up the Storage Hill or ILM?

If you had IPv6 fully implemented so that each disk drive had a unique IP address could you Implement ILM better?
Could you Manage it?
How?

There is a big difference

There is a big difference between the marketing effort and the technical effort. The marketing always has to change, hence it is good for everyone to come down on ILM and say it is dead, so the industry can change the label. On the other hand, much interesting technical work is underway that has the potential of changing the way we view storage systems and it is not just HSM.

What I am about is "Storage Service Management" and it happens that the words under ILM are now consistent with that goal. I can live with ILM or without ILM, but I can't not consider storage as a service. When we take a service view, with whatever label is on top of it, we provide a cost effective solution. That is how it started, before the marketing, and that is how it will end. As a service, the TCO is reduced by standardization, even without technology. The standard is only complete when process for managing storage is documented and the tasks turned over to operations. Then the storage tasks can be managed by fewer people in a predictable time with a predictable outcome at a predictable cost.

Now the reason people are running out of IOPS is that they do not know how to, or choose not to do a capacity plan. Somehow, they think that by buying a bigger box that it will fix the problem, or perhaps by using a higher speed network also fix the problem. There are scientific ways of calculating throughput in an array and on a channel, if you violate the rules, there is no miracle that can pull you out of the fire. These are important skills that an IT person should have and practice - it's just a ranting if you expect someone else to take responsibility, so you do not have to.

From a positioning standpoint, I do not sell anyones hardware. My business is assessment, planning and design, in addition to education. You can make anyones product work it is just a matter of knowing the numbers.

As it is obvious that you and I read the same content, you missed the obvious, that the need is to manage storage as a service that is coss-effective, reliable in performance with an acceptable level of availability - you could always do that, just most choose not to put in the effort.