IT shops seeking more accurate data warehouses

Companies are urged to “know their customers” if they want to build exceptional businesses with a loyal, satisfied customer base. But companies with multiple business lines operating in silos often don’t have the integrated information needed to discern macro patterns in their customers’ behaviour.

Decision-makers are often flying blind, making assumptions about their customers based on intuition, gut feel and other such voodoo methods.

Once they implement an enterprise data warehouse (EDW) to integrate the whole, many companies are surprised to learn many assumptions they’d made about their customers were wrong, said Stephen Brobst, CTO of Teradata, a division of NCR and major EDW vendor based in Dayton, Ohio. “This happens all the time,” he said.

But after all the blood, sweat and tears companies put into implementing an EDW, said Brobst, they are disinclined to publicize their “customer surprises,” out of fear of tipping off their competitors. Brobst offered a number of anonymous client stories to illustrate the types of insights brought to light when customer information is integrated across business lines.

They represent a broader cultural shift that’s underway, said Brobst — a shift that might be characterized as the triumph of science over art in business decision-making. “The old school used experience, gut feel, and intuition to make decisions. The new-school (members) are using sophisticated business intelligence tools, and they’re outperforming the old school.”

In one case, an airline discerned a correlation between meal preferences and show rates. Airlines typically use historical analysis to predict how many people will actually show up for their flights. This allows them to optimize the number of seats sold by over-booking flights by a certain margin based on their projections.

In the past, the airline used aggregate data based on seats sold in each class of service to predict show rates. After an EDW was implemented, the company used multiple variables based on rich, detailed information available from previously unexamined areas. A surprising finding was that vegetarians are the most reliable customers. “Ordering a vegetarian meal was the variable that was most predictive of a customer actually showing up,” said Brobst, adding that the sociological reasons underlying this quirky finding had not yet been determined.

In another example, an international banking firm developed a customer profitability model based on a value assigned to each of the products the customer owned with them. These values, in turn, were based on the average profitability calculated for each product.

After an EDW was implemented, the bank had detailed customer information available for analysis. Instead of using estimates (averages based on aggregate data) to determine profitability, it looked at a wider range of actual customer behaviours underlying each product.

The bank discovered that its previous model was significantly inaccurate. A variety of factors it had previously not considered affected a product’s profitability: whether credit card payments were made online or by cheque, if a customer made frequent calls to the call centre, and so on. “By using detailed instead of summary data to calculate profitability, about 75 per cent [of their customer profitability calculations] changed by two or more deciles,” said Brobst.

This detailed approach saved the bank from making a bad decision. It had planned to cancel a product, a direct deposit bank account for minors, because it was deemed unprofitable under the old model. When it looked at actual customer behaviour, the bank discovered that the cost to deliver the product was actually much lower than average, since customer service requirements are minimal: minors don’t use ATMs, withdrawals are infrequent, bill payment features unnecessary, and so on.

Brobst also offered a case from the manufacturing sector. Car manufacturers can’t test all parts for all conceivable combinations of conditions before selling their vehicles, so failures of non-essential parts are tracked after purchase. So one car manufacturer used its EDW to understand patterns in its parts failures

By analyzing detailed historical data, the company observed a correlation between the failure of a particular part, and the temperature of the region where the customer reported it. The pattern indicated that when the weather was particularly hot, the part failed.

Based on this information, the company was able to handle the recall in an orderly fashion, thus ensuring it had sufficient replacement parts on hand. Instead of doing one major national recall, the company used a phased approach, targeting customers in hotter U.S. states such as Arizona and Texas first, then targeting other states based on weather patterns.

George Goodall, a research analyst with Info-Tech, a London, Ont.-based IT research firm, agrees the trend towards evidence-based decision-making is positive but warns there’s a certain amount of voodoo in EDW too.

“There’s a real risk that when companies analyze their mountains of data, they will find correlations that are not in fact statistically significant,” he said, adding that this is the type of error that feeds such controversies as linking cell phones to cancer and power lines to leukemia.

QuickLink 057330

Related links:

How do data warehousing practices affect BI efforts?

Pondering the future of data warehousing

Related Download
Virtualization: For Victory Over IT Complexity Sponsor: HPE
Virtualization: For Victory Over IT Complexity
Download this white paper to learn how to effectively deploy virtualization and create your own high-performance infrastructures
Register Now