Taming data chaos

That motley collection of corporate data you’ve been accumulating probably resembles a Jackson Pollock painting, as my colleague Pimm Fox puts it so colourfully. Several drips over here, a few splatters over there, a big blob right here. It’s a mess: There are departmental data silos and data marts and data warehouses, not to mention those end-user spreadsheets. Every day, more and more data pours into the enterprise from Web clickstreams, consumer transactions and external data vendors. Meanwhile, the folks in marketing want to do sophisticated data mining and create a single view of the customer’s various contacts with the company.

Given this situation, it’s not surprising that the top two issues in data management are the following:

1. Integration: Merging, cleansing and standardizing data as part of an information quality program while integrating those data silos.

2. Scalability: Making sure the hardware, software and networks can handle increasing amounts of data and growing business demands without crashing or slowing to a crawl.

Those problems surface throughout this special report, which includes many tips from your peers on how to cope.

Now, I know that data integration and database scalability involve hard work, and it isn’t particularly glamorous work yet. But as companies begin to realize that data is a valuable business asset just as valuable as the other assets they manage, perhaps more so then data management will become a very high-profile function indeed.

In fact, if your company is implementing a CRM, ERP or supply chain system, then it has already (implicitly) decided that it’s in the data management business. Within the next two years, more business decisions will be based on electronic data, and more interactions with customers and suppliers will be handled by electronic data systems.

What could be more important than that?

Mitch Betts ( mitch_betts@computerworld.com) is director of Computerworld’s Knowledge Centers.