Industry


Ads by TechWords

See your link here


Is data quality what's next?

Yesterday I posted a note about what I think is the current state of business intelligence--companies have finally realized that it's an essential ingredient in business. 

In response to the post, Morgan Goeller added a thoughtful comment about what's next with BI.  In his opinion, it's information quality.  And I have to say that I agree with him.  And I've probably mentioned it in the past. 

Organizations seem to be collecting just about every piece of data that comes through their ports and connections.  The thing is not all of that information has any value at all.  In fact, in true pack-rat style, much of the information probably is collected simply as an effort to ensure that what's needed actually is kept. 

But these collect-everything-and-sort-it-all-out-later tactics are going to become very costly before it's all said and done.  Already, companies are stretched and expanding, trying to find ways to fit all of the information collected into storage spaces that are quickly growing too small.  So, what's in store for the future?

What needs to be in store is that companies take a step back and get a handle on the information they are collecting.  In my mind, it's simple (though admittedly, it's not quite this simple in the real world): Determine what information is useful, then determine the lifetime of that usefulness.  Then, if it doesn't fall into the useful category, don't keep it.  And when it passes the pre-determined usefulness date, get rid of it.

I'm sure it's not that easy, but this simplistic mindset needs to be applied to the way we collect data.  If it's not, it won't be long before there is far more data than we know what to do with. 

Data quality does need to become a focus for our business intelligence efforts.  Not to the extent that every other aspect of business intelligence suffers, but at least to the extent that the information that is being collected actually has some useful (and hopefully revenue-generating) purpose.  If it doesn't, then why keep it?

What People Are Saying

To Davids comment I'd add

To David's comment I'd add fixing processes responsible for data quality errors in the first place. Sustainable production of reliable BI requires we treat the causes of bad data quality not just the symptoms. BI is also about people, processes and rules not just routers, computers and wires.

Jerri- The only comment of

Jerri- The only comment of yours that I would challenge is your limiting the focus to only Business intelligence. In fact, as the quality of data that flows into a data warehouse supporting a BI program is actually sourced from upstream locations, the need to focus on data quality spans the entire enterprise, both in operational and analytical functions. Some failure points associated with BI activities are in the inconsistency of data (even if the data warehouse version is *provably correct*) with legacy report-generation applications.

Only recently has there been a growing recognition of how the quality of data is tied to business productivity, and I believe that this trend will continue!