BI cuts path through data jungle

Making sense of the immense piles of disjointed data amassed by government is paramount. Translating public spending into meaningful results is what earns it credibility. System integration, information lifecycle management and other initiatives to re-order information processing will take years to come to fruition. What to do in the interim with existing data and systems?

Some government agencies are turning to business intelligence (BI) tools that work with a variety of legacy systems to track their performance. The Canadian Department of National Defence (DND) recently implemented BI software provided by Ottawa-based Cognos Inc. to improve its financial reporting.

“We’re using BI software on top of our SAP system to take the financials and translate those into how much we’re actually spending on activities,” says John Scarlett, a senior change management analyst at the DND. “That way, we can explain to Joe Q. Public that this is what you’re getting for x-billions of defence spending.”

The data generated by the SAP system is accurate, but strictly financial, he explains. There is also a great deal of non-financial planning and management data outside the system, typically captured in ad-hoc spreadsheets or databases. To bring meaning to funding allocations, BI links the high-level financials within SAP to detailed activities captured without, by mapping the DND’s 5,000 cost centres to its 36 categories of activity.

“So instead of saying these dollars are going to this cost centre, you can say it’s going to army recruiting and training activities,” says Major John Pinsent of the financial decision support department of the DND. “We’re learning how to bring together data sources and finding the common linkages.”

This is achieved without reconfiguring systems or existing data, says Scarlett. “We’re not modifying the data from source systems. SAP and other data are fed into the data warehouse, and Cognos is the presentation layer that sits on top,” he says, adding that relevant information is accessed by the BI tool based on linkages to the right data and common business rules.

While BI doesn’t tackle data errors or redundancy directly, it does identify areas where data might not make sense, by throwing up red flags. “BI is the window into the data,” says Terrence Atkinson, director of public sector solutions at Cognos. “If you have one way of recognizing a client or record in one system and another way in other systems, that’s where you begin understanding the inconsistencies.”

The performance tracking capabilities of BI can also help bring clarity to information management and connect it to goals and outcomes, he says. “When you’re developing systems, you need perspective on the kind of information you need.”

Key performance indicators are derived from data that captures and measures results. For example, to track progress in child welfare, case managers will need specific information to understand the success of a program. “In building metrics and reports, that will drive the processes you use and the data you capture. You should be able to capture the different data elements that make up the metrics. This helps define the policy rules around data: what needs to be kept, for how long and so on.”

Unlike the commercial sector, government often needs to track many intangible factors over a long period of time to determine if a program is a success, he says. For example, if the goal is to lower the rate of teen pregnancy by one per cent, a measurable outcome, this means changing social habits, which are harder to define or track.

“Activities might involve going to high schools, running ads on TV and MySpace and so on,” says Atkinson. “It may take years before you see outcomes, and it’s hard to measure progress in the interim. Sometimes proxies are used: how many activities are underway and how many people were touched, with an eye to the long-term goal of lowering the rate.”

In the United States, BI is being increasingly used in the public sector to develop meaningful reports that can be generated quickly, driven by government mandates, he says. “The Government Performance Act requires each department to show how it’s performing in various programs. This means stating their expectations, showing how they’re performing against those and publishing this information on their websites.”

In Canada, there are similar mandates. “One requirement all departments have is to complete a program activities architecture that connects money spent to activities,” says the DND’s Pinsent, adding that Treasury Board has recently made the integration of non-financial information in reporting a requirement.

At present, BI tools are designed to work with structured data organized in databases, says Atkinson. “But a major trend in the public sector is interest in unstructured data,” he says, pointing out there’s a lot of untapped knowledge in e-mails and other unstructured documents.

Vendors are working on developing Google-ish search engines for organizations, he says. “Cognos doesn’t report against unstructured data, but we have partners who offer search engines that go into documents and tag them, and this allows us to read the tagged information in our reporting.”

These are still early days for this next-generation capability, but this is clearly the way BI is evolving, he says. “Organizations want to get access to all relevant information. Users don’t care if it’s structured or unstructured; they want access to it all.”

Rosie Lombardi is a Toronto-based freelance writer. She can be reached at

Related content:

BI tools help PHAC whack infectious disease

Integrating IT for collaboration

The predictive physician

Related Download
Improving the State of Affairs With Analytics Sponsor: SAS
Improving the State of Affairs With Analytics
Download this case study-rich white paper to learn why data management and analytics are so crucial in the public sector, and how to put it to work in your organization.
Register Now