It’s hard to go full steam ahead if you’re not sure that your ‘engine’ is in peak performing condition. Prior to this year, that was the problem facing Information Services management at the Canadian Pacific Railway (CPR).
Although several measures were in place to gauge various aspects of I.S. service delivery, project management, client satisfaction and people management, the I.S. management team didn’t have access to a comprehensive, integrated measurement system to tell them how they were doing in a variety of important areas.
In order to remedy this situation, one of CPR’s key programs – initiated in January, 2000 – was the development of an ‘I.S. Dashboard’, which would allow these leaders to gauge their performance on important management dynamics, monitor trends, take corrective actions when appropriate, and communicate results to all stakeholders.
According to Allen Borak, Vice President, Information Services at CPR, “What we needed was an effective, visual means to distill information on how we were doing; something that would integrate the higher level, strategic dimensions with the day-to-day operational performance”.
The CPR Information Services environment is indeed complex. The company employs about 700 I.S. professionals – both employees and contractors. Geographically, service delivery is highly dispersed with several essential applications needed at remote locations across Canada and in the United States. About ten of these applications are critical to the movement of trains, and, therefore, to the revenue and profitability performance of the company. MVS, Unix and NT platforms combine to support over 8,000 workstations spread throughout metropolitan and remote locations.
By utilizing Balanced Scorecard principles, the leadership team developed the following guidelines:
- The I.S. Dashboard must be ‘balanced’, demonstrating all the important dimensions of running an information technology environment;
- The business must be able to understand and appreciate the measures used;
- The Dashboard should be simple – with fewer, rather than many, dials;
- For most measures the short- and long-term trends may be as important as the current value.
Further, it was determined that this ‘instrument’ would evolve, and measurement systems would get better as experience was gained. But it was important to implement the first phase quickly.
Overview Of The I.S. Dashboard
Conceptually, the Dashboard demonstrates performance along two dimensions. Said Borak, “We have to demonstrate that we are good at ‘blocking and tackling’ our day-to-day performance; these measures had better be positive, or else. In addition, however, we need to show that in the longer term we can add value to the railway and that we are a strategic resource.”
CPR’s I.S. Dashboard consists of five main dials answering key questions as follows:
I.T. Leadership: Is I.S. working on the right things at the right time to improve CPR’s competitive position, asset utilization and cost structures?
Project Management: Does I.S. consistently deliver quality products (e.g., new applications and infrastructure components) on time and on budget?
Service Delivery Management: Are the day-to-day services from I.S. reliable, responsive, well supported and cost effective?
People Management: Does the leadership team guide and support well, so that I.S. professionals can deliver results and achieve personal and career goals?
Client Satisfaction: How satisfied is the business community with the business applications, corporate services and support delivered by I.S.?
Physically, the Dashboard is web-enabled, having been developed through Microsoft FrontPage. “This allowed Information Services to taste its own medicine and gain some valuable experience with this new development technology,” said Borak.
Figure 1 shows the five main dials that are published monthly, along with the last update of the dial, the dial score during the last three reporting periods, and the longer-term trend of the dial.
By pointing and clicking on any dial, the component dials are displayed, and eventually, through further drill down, the raw data itself can be exposed, whether it be on an Excel spreadsheet or a PowerPoint presentation. Many of these measures were already in place at CPR – it was simply a matter of grouping them into categories that made sense to the leadership team, and coordinating their publication through an I.S. ‘Dashboard Coordinator’. Figure 2 shows the results of a ‘point and click’ on the Service Delivery dial.
The component dials are set according to the actual attainment against a predetermined target, whether this target be the percentage availability of an application or the targeted outcome of a senior management or client survey. This results in a red, yellow and green colour-coding scheme for all dials, with each of these further subdivided into three categories: low, medium and high. These colours, with their three varying shades, can be interpreted as follows:
- Red requires immediate (or continued) management attention and involvement;Yellow requires management attention and investigation (the degree of management rigour depending upon the low, medium or high rating);Green indicates acceptable or above standard performance.
As a rule of thumb, attaining a targeted performance level merits a ‘low green’, while over-attaining the target merits a medium or high green. Under-attaining the target will merit various shades of yellow or red, based on the severity of the shortfall.
These calculations have all been programmed using FrontPage, through a series of algorithms and table look-up scenarios.
The Meaning Of The Dials
I.T. Leadership
The I.T. Leadership dial consists of two sub-dials:
I.S. Strategic Alignment (50% weight)
- An appraisal based on structured interviews with a selected subset of senior managers, designed to determine I.S. engagement in determining strategies and business plans, and the alignment of I.S. strategies with corporate direction.
I.T. Value Contribution (50% weight)
- A return-on-investment indicator designed to demonstrate, in quantitative terms, the value of the contribution of information technology to the business.
For the ‘I.S. Strategic Alignment’ indicator, senior management interviews, using consistently applied questions, are conducted semi-annually, resulting in a composite rating scheme from one to seven. This sub-dial is then set based on the percent attainment against a target of 5.0 as a result of these questions.
The ‘I.T. Value Contribution’ sub-dial is established quarterly, based on an analysis of the financial profiles for new projects that have passed their Definition Phase and have approved business cases. This dial only evaluates new discretionary investments made by the business in I.T. It assumes that all previous investments for the current production applications and infrastructure have been justified in the past. The business case for each project spans four years and the cumulative benefits and costs for all business cases are analyzed to determine a composite Internal Rate of Return (IRR) value. This, in turn, is compared to a 15% target, deemed to be a reasonable expectation of what a business would achieve from a technology-based investment. The dial setting is determined on the IRR percentage against this target of 15%.
Project Management
The ‘Project Management’ dial consists of two sub-dials:
On Time / On Budget Performance (60% weight)
- On time / on budget performance, as determined by the cost performance and schedule performance indicators (CPIs and SPIs) published by the company’s I.S. Project Management Office.
Quality (40% weight)
- Project quality, as determined by the number of emergency fixes for new applications required each month.
The ‘On Time / On Budget’ sub-dial is based on a target of 80% of the projects with both the CPI and SPI greater than or equal to 1. If less than 50% of the projects have both CPIs and SPIs greater than or equal to 1, the indicator slips from yellow to red.
The target for emergency fixes per month was determined following an analysis of historical error rates; it is 16 emergency fixes per month. A ‘low green’ is attained for this number with higher shades of green for lower emergency fix rates, and yellow or red ratings for higher fix rates.
Service Delivery
The intent of the ‘Service Delivery’ dial is to measure two important dimensions of the impact of I.S. services:
- Does I.S. maintain service level targets for critical services where failure results in severe business disruption, and how quickly does I.S. restore service for these critical areas?Do the I.S. services encourage business productivity through prompt responsiveness to problem situations?
Hence, the ‘Service Delivery’ dial consists of three sub-dials as follows:
Availability (35% weight)
- ‘Applications availability’, measuring the performance of ten critical applications against their targeted availability (60% contribution).’Infrastructure availability’, determining the stability of the computing and communications infrastructure, critical to running day-to-day business, against targeted availability (40% contribution).
Service Restoration (15% weight)
- ‘Severity One incident recovery’, measuring the responsiveness of I.S. in restoring service to critical incident situations against a targeted response time (100% contribution).
Support Delivery (15% weight)
- ‘Call answer rate’, measuring the percentage of Help Desk calls answered within two minutes and comparing this to target (66.7% contribution).’Abandoned call rate’, measuring the percentage of calls abandoned by users in the queue and comparing this to target (33.3% contribution).
Cost Effectiveness (35% weight)
- ‘Year to date’ actual / budget ratio attainment for operating costs incurred by I.S. (100% contribution).
People Management
The ‘People Management’ dial is composed of three sub-dials, each counting for one third of the weight, as follows:
- ‘Retention’, measuring those factors which contribute to retaining those employees who meet or exceed performance expectations.’Job capability’, measuring those factors that contribute to a productive environment.’Commitment’, measuring the motivation and commitment of employees to contribute to I.S. and CPR success.
These measures are determined from specific questions answered through personal surveys administered to specific populations of the I.S. community throughout the year, along with recorded data, such as the resignation rates among employees with satisfactory, and above, performance appraisals.
Client Satisfaction
The ‘Client Satisfaction’ dial is based on:
End-User Feedback (80% weight)
- ‘End user feedback’, as determined by the percent attainment against target from the semi-annual Customer Satisfaction Survey.
Senior Management Feedback (20% weight)
- ‘Senior management feedback’ on a questionnaire concerning service delivery, administered semi-annually.
The Senior Management input is important to gain a balanced perspective between the level of service being delivered and the affordability of that service.
Both ‘Client Satisfaction’ sub-dials are set on the percentage attainment against the target scores (on a scale of 1-7) as determined by the leadership team. Currently, the Customer Satisfaction Survey targets a 5.0 rating, while the senior management feedback strives for a 5.5 rating.
The Road Ahead
CP Rail’s I.S. Dashboard is already having an impact on I.T. at the company.
“The process of building the Dashboard has caused us to step back and understand what was important in running our environment, both to ourselves and to our external stakeholders,” noted Allen Borak. “It led to a few healthy debates among members of the leadership team”.
The Dashboard will not be a static instrument. It will evolve and change with technology, along with I.S. planning and delivery techniques. It promises to be an excellent tool to help the company, at least from an I.T. perspective, keep its eyes on the (rail)road.
Graham McFarlane is a Management Consultant and a Director of Western Management Consultants in Calgary. He practices in the management of information technology, and led the design and implementation of the I.S. Dashboard at CPR. Mr. McFarlane is a Professional Engineer, an Information Systems Professional, and a Fellow of the Canadian Association of Management Consultants. He can be reached at [email protected].
CP Rail – Past and Present
CPR has come along way since its founding in 1881, when it helped open the Canadian West through the building of a transcontinental railroad. Today CPR provides rail and intermodal freight transportation services over a 14,400 mile network extending from Montreal to Vancouver, and throughout the American Midwest and Northeast. It also serves ports on the East Coast and West Coast, and is the leading carrier in the intermodal industry, with 24 terminals across Canada and the Northern U.S. The company moved 192 trillion “ton-miles” of goods in 1999, earning $762 million on gross revenues of $3.5 billion.
From 1997 to 1999, CPR pursued an aggressive asset renewal program, spending more than $2.8 billion in that time period. The locomotive fleet has been rejuvenated with high performance technology that allows CPR to haul more freight, more efficiently with fewer locomotives; the intermodal system has been modernized; and an innovative and radically new way of moving trailers between cities – allowing trucks to drive on and off “continuous platform” trains – has been put in place.
In addition, significant investments have been made in leading-edge IT to enable business change. Senior Management anticipates performance improvements in three key areas: cost management, asset utilization and customer service. Several applications, critical to the efficient movement of trains and freight, have been built by Information Services over a number of years. CPR is also moving forward aggressively in the e-Business arena to enhance its portfolio of customer services and business channels.
All this positions the company well for the future and allows it to aggressively pursue its mission: “Through teamwork we will create value by delivering superior customer-
focused transportation solutions”.
The Balanced Scorecard Revisited
In the early 1990s, Robert Kaplan and Dick Norton published their ideas on using a ‘Balanced Scorecard’ to gain perspective on corporate performance. They contended that the publication of quarterly and annual financials was not enough to gauge the long-term health of an organization; other measures had to be considered.
This concept was important in the design of the I.S. Dashboard. The objective is to look at some of the longer-term performance implications of I.S., rather than focusing purely on today’s service levels and budget/actual status.
The Scorecard dials suggested by Kaplan and Norton looked not only at financial factors, but also at operational effectiveness, customer satisfaction, and innovation. The Balanced Scorecard attempts to measure the potential for sustained performance for an organization, and these precepts are reflected in the I.S. Dashboard at Canadian Pacific Railway.
