Measuring performance in Canadian health care

In health care, the continual improvement of both the quality and delivery of patient care is paramount to all other goals.

Consequently, hospitals across the country are seeking ways to help staff work more efficiently and productively. They are doing so under tighter budget constraints than ever before, and amid work by the Health Council of Canada to achieve a fully interoperable electronic health records system for all Canadians by 2010.

Business intelligence in health care

Most people agree that arming doctors and health care professionals with complete patient histories and performance data will improve and even expedite care. However, Canadian hospitals need to balance cost, privacy concerns, external accountability, the need for accurate patient information and quality improvement imperatives.

The challenge is how to accomplish this delicate balance in an industry that has historically been paper-based and where information is often fragmented and housed in disparate locations.

Accountability versus quality improvement

In broad terms, it is possible to distinguish between two roles for performance data in the public sector. The first relates to discharging accountability relationships: that is, giving account to funding bodies, ultimately the taxpayers of Canada. The second role relates to quality improvement: an internal management role.

This involves the use of performance measurement data to monitor, analyze, prioritize and learn. In the province of Ontario, it appears that important advances have been made with respect to external accountability, while quality improvement – systemically using data generated to improve performance – lags behind.

Perhaps the most comprehensive example of performance measurement in Ontario health care is the Hospital Report series, a joint initiative of the Ontario’s Ministry of Health and Long-Term Care and the Ontario Hospital Association.

Since 1997, hospitals throughout Canada have been involved in editions of the Hospital Report, originating with the publication of a systematic report on acute care in 1998 and subsequently extending into other areas, including emergency department care, complex continuing care, mental health, and rehabilitation.

These are comprehensive, annual, province-wide hospital scorecard reports based on a balanced scorecard methodology and comprise measures of financial performance, patient satisfaction, clinical utilization and outcomes, and system integration and change.

Together, they represent one of the largest balanced scorecard efforts in North America, in terms of scope, that has ever been made public. They also represent one of the largest surveys of patient satisfaction ever conducted. The Hospital Report series has focused health care attention and made an important contribution from an external accountability point of view.

To this end, close to nine of every 10 Canadians think that publicly reported hospital scorecards are a good idea. However, research has consistently found a poor awareness of and access to these reports from within health care organizations; a lack of resources, time and support to understand, analyze and respond to the data; and dissatisfaction with the definition of many of the performance indicators.

Perhaps most alarmingly, survey research suggests that between 35 per cent and 50 per cent of frontline workers and mid-level managers within Ontario hospitals are unaware of the initiative.

Taken together, this suggests that while the public release of comparative data may help focus attention on the quality agenda within health care providers, greater efforts are needed both to develop internal systems of quality improvement and to integrate these more effectively with external data systems.
The need for new technology

More hospitals are looking at implementing advanced business intelligence (BI) tools to address these challenges within the health care sector. Business intelligence refers to a group of tools that collect and analyze internal and external data to generate knowledge and value for an organization.

Used effectively, these tools can help people to make better business decisions. A BI system is designed to interpret data in a way that will help, for example, to integrate initiatives, align organizational units and resources, and improve performance.

The ability to improve patient care in Canada is closely tied to the availability of sophisticated tools that help to improve the dissemination of information across the user interfaces that health care employees use every day.

Effective BI tools also offer the potential to improve efficiency by reducing the costs associated with errors, data re-entry and document processing and storage. In turn, BI tools promise to improve management by providing feedback and promoting planning, prioritisation and cross-institutional learning.

Strengthening accountability has been a key theme of most proposed health care reforms. However, it could be argued that the emphasis of performance measurement reforms should increasingly be placed on internal quality management rather than external accountability.

Investment is required in human resources and information technology that is aimed at assisting quality improvement efforts. Recent research suggests, for example, that Canadian hospitals spend approximately two per cent of their budgets on IT, compared to 5.5 per cent in the U.S.

Furthermore, performance measurement is often considered within the purview of those in the quality management section of a hospital’s organizational chart. The quality program, if indeed it exists, is often located within a clinical portfolio and is frequently under-resourced, in terms of expertise in quality improvement processes. Rarely are the clinicians and quality improvement workers meaningfully associated with the core analytical expertise of the hospital.

It is important to look at middle managers and frontline staff within hospitals and to consider what challenges they are being faced with, the sort of information they would like, the format they prefer to receive it in and how this might actually result in an improvement of overall performance management.

Effective performance measurement needs to involve both senior leaders who understand the long-term goals and frontline personnel who appreciate the kinds of measures needed to support local decision making.

Data standardization

Unfortunately, given the fragmented nature of communications and the relative paucity of digitized clinical records in Ontario, gathering information, even something as simple as a patient file, can pose a challenge.

The Ontario Hospital Association and Ontario Buys, a program within the Ontario Ministry of Finance, are committed to ensuring the broad dissemination of information across the health care sector. This is a step in the right direction.

However, a similar commitment to technology that can help achieve quality improvement goals and improve patient care processes is less forthcoming. To help meet this commitment to technology, the health care sector needs to sharpen its focus on performance measurement and delivering the right information to the right person at the right time, emphasising how individual employee actions affect results.

Universal health care in Canada is a burgeoning and polemic area at a critical juncture in its history. Michael Moore’s recent film, Sicko, reflects the international gaze on the sector and perception studies continue to find that timely access to health care is one of the highest policy priorities for Canadians. Performance measurement and reporting in the sector have clearly made advances in the past decade, but more needs to be done to continue to move the sector forward.

By implementing IT systems based on a commitment to BI technology, accurate, standardized information is likely to be more readily accessible and understandable to health care professionals, offering the promise of improved patient care, efficiency and management.

Clinton Free is an assistant professor at the Queen’s School of Business. Contact him at

Related content:

Scotiabank aiming to master data management

BI cuts path through data jungle

Integrating IT for collaboration

The predictive physician

Health integration calls for more collaboration

Related Download
Improving the State of Affairs With Analytics Sponsor: SAS
Improving the State of Affairs With Analytics
Download this case study-rich white paper to learn why data management and analytics are so crucial in the public sector, and how to put it to work in your organization.
Register Now