Wondering how the rest of the organization views the work of the IT department? Well, as the saying goes, all you have to do is ask.
That’s what pharmaceutical firm Eli Lilly Canada Inc. does – regularly. CIO Hugh Sawyer says his department has made it a habit to leave simple survey cards with users whenever it installs new equipment. They fill them out and return them via inter-office mail. Periodically, Sawyer does a similar survey to see how happy users are with the support they get from the help desk. And then there’s face-to-face contact with the heads of user departments. “I meet with the directors almost every month, ask them about their business, what are the current issues, what are their worries, how is IT doing – more of a subjective assessment, and I find that one perhaps the most valuable.”
The point is, Sawyer knows that to get the information he needs to do his job well he must take the initiative, and so he solicits feedback, routinely, from a variety of people and in a variety of ways.
Things are much the same at the Bank of Montreal. It too, for example, relies heavily on personal contact. Systems department personnel work in customer units and are part of the management teams there, so they have an inside track on those people’s needs and concerns, says Barry Gilmour, Executive Vice-President of Systems.
The bank does formal surveys as well, particularly after a project is completed. Two or three weeks after a project finishes, the bank surveys those affected to see how happy they are with the results, whether IT delivered what it said it would on time and on budget, and whether users would like to see any changes or enhancements. Once or twice a year the bank does a wider survey of users to look at broader IT-related issues.
“Customers definitely feel better when they feel that they’re influencing IT one way or the other,” says Gilmour. IT staff meet regularly with customers to discuss concerns raised through mechanisms like surveys. If that’s not enough to ensure that systems staff take surveys seriously, maybe this is: the results of these surveys are a significant part of the formula that determines how much money there is in the pot for IT staff bonuses at the end of the year.
STRATEGIC SURVEYS
Leslie Knox, a partner with Deloitte Consulting in Toronto, says IT surveys of business needs can take two forms: tactical and strategic.
The tactical survey usually has a specific purpose. It might be done to help in drawing up a service-level agreement, for instance, or to determine whether system uptime is adequate. And, of course, tactical surveys can spot specific problems that might otherwise take awhile to surface.
The more interesting surveys, though, are those undertaken for strategic reasons, and focusing on the big questions, such as: what does the business want to do, and how can IT help it to do it?
Knox says surveys like that can do a lot to help information technology units stay in tune with business goals, provided they are done right. “Typically, the end product is some form of an IT strategy and an IT plan that maps out what IT is going to deliver, usually over a period of a year or two, or however far out the business wants to think.”
Because they tie in with business goals, good strategic IT surveys must be done at the right time, most often as the next step after the organization has reviewed its overall strategy. If that happens annually, then a process of finding out how IT can help meet the strategic goals should follow close on its heels. Some organizations review their business strategies more often than that, Knox notes, sometimes just to fine-tune priorities. Again, IT should follow the business’s lead. At the least, she says, the IT strategic survey should be repeated about once a year.
Another good time for a survey is when the IT department is gearing up for a change, such as a reorganization or adding staff or services, says David Smiderle, a Toronto-based consultant with Hay Information Services Group who specializes in conducting IT effectiveness surveys. He also believes regular follow-ups are important, and suggests a more formal survey once a year and informal versions at three- or four-month intervals in between.
The survey process can be a lengthy one. When an organization first does it, it often takes two to four months. “What’s time-consuming in the first cycle is doing the assessment really well and understanding where you are – that’s half the time,” Knox says. Subsequent surveys might be done in three to four weeks.
GETTING THE RIGHT PEOPLE INVOLVED
After you have figured out when to survey, the next critical question is who – not just who to survey, but who to do the surveying. The ideal choice, Knox says, is “a different type of IT person than the norm.” It should be done by those with a thorough understanding of the business, who can talk in the language of the business. Otherwise, he warns, “you have technical people trying to have conversations with business people, and that just ends in frustration.”
Gilmour agrees. “Years ago it was mostly technologists that were in this business,” he says, “and as the years go on, what you’re seeing is an increasing need for the ability to relate to the business.”
And who do you talk to? Start with senior people, Knox advises – the more senior the better. It’s from these people that you’ll get the big picture. What is the organization trying to achieve? What are its objectives for the next couple of years? What factors will be critical to success? How can current IT processes support those objectives? What needs to change?
“Starting with the senior people is a good way to get the lay of the land,” Knox says. “Once you understand what the objectives of the business are, operationalize that and try to understand what you need to do to meet those objectives.”
Eli Lilly Canada has an IT prioritization committee, which includes CIO Sawyer and senior members of his IT team as well as directors and associate directors from other departments. Sawyer pulled together the committee because at one time he dealt solely with senior management, and was accused of doing arbitrary projects. He says the most fun he has with this committee is when he says, “Look, we’ve run out of money and we have to start to prioritize.”
SURVEY CONTENT
The first phase of a strategic survey should be done through one-on-one interviews. Then, as you begin to flesh out the strategic goals with the details of how information systems need to evolve to support those goals, you will want to talk not just to senior people but to people at all levels of the organization – to those who will have to turn the goals into reality. At this stage, focus groups or written questionnaires can play a role.
Of course, you need to think about what questions to ask. Ross Stockwell, whose Toronto-based company Ross Stockwell & Associates specializes in conducting employee surveys of all sorts, offers the following advice: “If you’re going to do a survey there needs to be some consideration of what you’re going to do with the results.” That consideration should be up front, because to some extent it drives what’s going to be in the survey.
“The technology of doing the survey is relatively straightforward,” Stockwell says. “The difficulty is in designing it and getting the political buy-in and support. Most of the work in these surveys comes in just getting the executive group on the same page.”
Smiderle at Hay Group says one way to make sure you are asking the right questions is to involve users in the design of the survey itself. Ask them what the issues are that concern them, and how you can incorporate them into your questionnaire. Of course, there may be issues of concern to IT people that need to go in as well. Some likely issues are the overall effectiveness of IT, how responsive IT people are to user needs, and the clarity and accuracy of the information they provide.
Questions need to be specific. Sawyer at Eli Lilly says he has more or less given up on asking open-ended questions like “How is our applications support?” The results are too inconsistent, he says. That’s because expectations can vary widely from department to department. One department might say applications support is excellent because the IT group keep major systems running smoothly, while another might consider it terrible because they can’t get help creating custom spreadsheet models, which IT doesn’t consider part of its mandate.
“Customer service is based on expectations,” says Philippe Denichaud, a psychologist who works with IT departments and others through his Toronto-based company Breakthrough Management Inc., “and it’s an emotional journey rather than a practical journey.”
Though it may be useful to ask respondents to rank a particular service on a numerical scale, a much better picture can be obtained by first asking how important the service is. That helps put the answers in context and lets you set priorities. Additionally, you should ask for comments as well as simple rankings. That gives people an opportunity to tell you what in particular they aren’t happy with or would like you to do differently.
Intangibles are not usually very useful, Denichaud says. If a user says an IT person has “a bad attitude, what does that mean? It might mean the person, under pressure because a high-ranking executive was looking impatiently over his shoulder, did a poor job of explaining a technical problem.”
Depending on who you are surveying and what kind of information you are seeking, it may be a good idea to keep responses anonymous. “People have to feel that by answering or being involved in this they won’t somehow be penalized,” Smiderle says. That’s an argument in favor of a written questionnaire rather than face-to-face interviews or focus groups in some cases. But anonymity is less important in surveying senior people, who are less likely to feel they might pay a price for saying something you don’t want to hear.
RESPONDING TO THE RESULTS
And what if people do say something you don’t what to hear? Are you ready for what you might find out? It’s important to know how you will respond if you get slammed, Stockwell says. If you find people are dissatisfied, do you have the will to do something about it?
The following advice should be obvious, but all too many organizations fail to heed it. It can be summed up in three words: use the results. Not only should you make sure you act on what your research has told you, but it’s important to let the people who were surveyed know that you’re doing so.
“People are often very gung-ho about doing the survey, but very rarely do they think about how they are going to deliver the feedback. That’s where I see a lot of mistakes being made,” says Stockwell.
He adds that 80 per cent of all surveys are not properly followed up. And that could cause problems. At this point you’ve raised the expectations of your customer group that something’s going to be done. If you go that far and then don’t follow through, doing the survey does more damage than good.
The IT department has to do more than just read and react to what a survey tells it, according to Deloitte’s Knox. The key to success is to change IT itself – switch its orientation to thinking about the goals of the business, rather than about technology for its own sake. And while that kind of thinking should to an extent permeate the department, she says, you need specific people in IT whose job is to be a liaison with other business units to help keep IT in tune with their goals.
Of course customer surveys will sometimes turn up problems or desires that you just can’t address, or can’t address right away. You should still acknowledge the concerns, even if the best response you can give is “we can’t fix that right now but we hear you.” Hopefully, users will understand and accept your explanation. If not, at least they know you heard what they said.
Grant Buckler is a freelance writer specializing in information technology and IT management. He is based in Kingston, Ont.
