The Annotated Feature is an ongoing series where we ask a Canadian CIO to offer their comments on a story from one of our U.S. publications. In this installment we have Harry Turnbull, executive director of IT for the City of Windsor, Ont., offer his take on what's right in this piece, what's wrong and what was left out.
Lies my vendor told me: Tech relationships gone wrong
By Cara Garretson
Three years ago, a luxury retailer with a growing online business faced a nightmare situation: The order fulfillment service provided by its SaaS vendor for online transactions went down for two days during the critical Christmas season. The retailer's website couldn't accept orders for a 48-hour period, resulting in frustrated customers and lost business.
In an attempt to mend relationships, "we ended up shipping big gifts after Christmas to customers [who were affected]," says a former senior IT manager with the retailer, who asked that his name and the name of the company not be revealed.
The relationship between the retailer and the SaaS vendor, which had provided the company with order fulfillment services for four years, was irreparably damaged.
Although the retailer deals in luxury goods and therefore is not a high-volume business (2,000 transactions would be considered a busy day), it had grown from roughly $10 million in sales when it signed on with the SaaS provider to about $50 million when the outage occurred, leaving the IT manager to surmise that the vendor's platform couldn't scale as advertised to handle that growth.
But the manager and his company will never truly know the cause of the outage.
Before the client could terminate the relationship, the SaaS provider acted first, asking the retailer to get off its platform within six months. The only explanation offered by the vendor was that it couldn't provide the type of "white-glove service" the retailer required.
Harry Turnbull: This is a great example of deflection, one of many tools vendors use to avoid taking responsibility for a situation, blame the customers “high” expectations. Contracts and Service level Agreements should include expectations of growth and clearly set the “high” expectations we have for those who are the caretakers of our information. This tool is used by vendors almost as often as finger pointing.
"It was a very ugly situation between the vendor and us. There wasn't much [either side] could do regarding repairing the relationship," says the IT manager.
Sweet turns quickly to sour
Relationships between technology vendors and their customers often start out sweet and turn sour, thanks to unmet expectations, half-truths, or products and services that simply don't work as advertised.
While it's hardly unusual to have strained buyer-seller relationships in any industry, dealings with technology vendors can be particularly thorny because of the complexity of the products -- you simply can't tell without an in-depth evaluation whether a complicated ERP package or networking equipment will meet your organization's needs.
Harry Turnbull: You cannot overstate the importance of the upfront work. There are very few easy IT projects, but they can be a lot smoother for everyone if you do your work upfront to ensure you are buying the right product and fully understand what is involved in implementing and supporting the solution.
What's more, IT products comprise multiple components -- hardware, software and, often, customized coding to fit the customer's environment -- making it difficult to pinpoint where the issue lies when problems arise, and which product or vendor is responsible.
"Vendors, like people, always present themselves and their products in the most flattering, if not most accurate, light," says Charles King, principal analyst at Pund-IT, an industry analysis firm. "The sheer complexity of some IT solutions creates opportunities for easy flimflams, as does customers' ignorance."
Harry Turnbull: Internal relationship building is critical to avoid this situation. Getting your internal customers to know that they should not take the canned demos and glossy marketing materials at face value will only happen when they trust that you are working in their best interest.
Even the metrics used to judge what should be straightforward features like performance aren't always terribly helpful, King says. Just one example: "While industry benchmarks like TPC-C for server transaction performance is an interesting measurement tool, it doesn't reflect commonplace data center or work scenarios, so it's not especially useful for determining the true value of a given system."
In the case of the luxury retailer, the IT manager felt that the SaaS provider misrepresented its transaction processing capabilities. "The service provider said no other client [than us] had this issue. We always suspected some overall volume problem; maybe we were on some shared platform and another client took it down so there was a domino effect," the IT manager says.
Harry Turnbull: I have heard that line “no other client had this issue” so many times. This can be difficult for you to verify so a great way for the vendor to start you down the road of actually believing this is your fault. Many products do have online forums associated with them and this is a great audience to find out who else has this issue and allow you to turn the discussion in your favour.
Experts who lack expertise
How do you avoid technical and interpersonal difficulties like those experienced by the luxury retailer?
IT managers say that being as rigorous as possible before any contracts are signed -- asking questions, verifying claims and getting promises in writing -- can help avoid conflicts and disappointments once the ink is dry.
Harry Turnbull: It takes hard work to get all your questions answered up front, but the ability to avoid situations where the vendor has mislead you, provides significant value. Unfortunately we tend to learn the hard way, we have to get burned before we implement more rigorous up front processes. Even when things are going well, then people start to question how we can streamline this process. This carries a risk of allowing the old habits to creep back in. Constant education of the process for our clients is critical.
The Daily Herald, a publisher in Everett, Wash., recently ran into a situation where more verification earlier on could have saved some headaches. As it was, misleading information provided by a vendor during an evaluation process resulted in significant extra work for the IT department.
The company, which puts out a daily newspaper, five weekly publications and one monthly publication, was searching for a new editorial system to handle a variety of tasks, including managing photos, text, graphics, story plans, assignments and notes.
The company asked a vendor how its software would run at multiple sites, since that was one of the prime requirements, says Michael Lapham, manager of technical services at The Daily Herald. The vendor said it could run Citrix at remote sites, and that it had in-house expertise to help install and configure remote systems.
After signing a contract with the vendor, Lapham quickly discovered that its Citrix expertise was exaggerated, to say the least.
"It ends up they didn't have any in-house knowledge of it at all, only that one of their customers was using it," Lapham says.
The vendor knew that remote access could be done with Citrix -- because another customer was doing just that -- but it had zero experience installing and configuring such a system and didn't offer any help. "So you could say what they told us was a bold-faced lie," Lapham sums up.
The Daily Herald's IT department had to do the Citrix installation, which ended up being a quick-and-dirty job just to get the system up and running. "We put in a minimal configuration; it wasn't exactly best practices. We're aware that we didn't end up doing it the best way, but that's what we had to do, since the vendor had nothing to provide us with," Lapham says.
Harry Turnbull: While internally I may sometimes state to our clients that vendors lie, what I mean by that is when a vendor says “yes we can do that”, quite often they neglect to include the part about, with a lot of hard work, consulting expense, possibly additional technologies and oh by the way, this is not what our product was designed to do.
Upselling and overbilling
Due diligence is ideal but not always possible, especially with smaller companies that don't have dedicated IT staffs and may not know which questions to ask.
A few years ago, IT professional Thomas Peacock got so fed up with the new president at the small IT services company where he worked in Spokane, Wash., that he quit four months after the new executive came on board.
The president "would tell a small business of maybe five users that didn't have particularly critical data that they needed a firewall. Then he would take an old computer that had been discarded and was ready for recycling, load it up with firewall software and sell it to the small business for $500," recalls Peacock.
"This firewall would require tending to once a month or so, which would generate an hour or two in service calls; even if the call was 15 minutes, he charged $100 for the hour." Since most of the people the company sold to didn't have technical expertise, it was hard for them to argue with the president, because they needed his company's support to keep the product running, says Peacock, who is currently looking for work.
Harry Turnbull: If we were building a building, it would be standard practice to have a third party review the designs and the commissioning. While there is additional cost to this concept, applying it to technology, in many cases could result in overall savings. The cost of the third party set of eyes could be offset by the reduced cost of the solution implemented because it now meets your needs rather than paying for functionality you will never use.
Bypassing the IT department
One way vendors avoid having to wrangle with IT is to go around the group entirely, says Mike Paradis, former CIO of Deutsche Bank Alex. Brown.
Technology companies have long seen the value in selling directly to the line-of-business managers, says Paradis, now an executive coach at CPI/New Options Group in Sparks, Md.
In heading up remediation programs for Y2K compliance in the late '90s at Deutsche Bank Alex. Brown, Paradis discovered that different business units had purchased and installed software without the IT department's knowledge or approval -- and now it wasn't Y2K-compliant, he recalls.
In another job, as database administrator for a large financial services company, Paradis at one point found himself having to support six different database packages to suit the needs of various systems purchased by line-of-business managers.
"Salespeople are very good at convincing business line people," he says. "They sing and dance, and the next thing you know, the business line has a product it can't support."
To combat that disconnect, Paradis suggests that both line-of-business managers and IT be involved in the evaluation and purchase of new systems, with IT taking the lead to ensure that the product will live up to its claims technically and integrate with existing technology.
Harry Turnbull: Getting internal buy-in across your organization so all levels know to come to IT with your issue and then let’s work together to find the right solution. Ongoing education, relationship building and trust that IT is there to help are the keys to build this culture and thwart attempts at end runs by vendors.
Canceled contract, personal attacks
Sometimes buyer-seller relationships are so strained that it becomes personal.
Eric Prosser had been CIO of Tulare County, Calif., for two years when it came time to renegotiate a 15-year-old contract with the service provider that was managing roughly half the county's IT resources. The outsourcer was angling to take over management of the entire county's IT resources, which would up the contract's value from $8 million to as high as $15 million, Prosser says.
While it would have made his life significantly easier to expand the contract so he could manage only this one outsourcer, he was not convinced that such a deal was the best thing for the county.
What's more, the outsourcer's proposed new contract -- which included technology such as VoIP to lower costs for the county -- ended up including fewer actual services for nearly double the money.
Harry Turnbull: While a contract helps set expectations for the relationship and provides a safety net for both parties, the real key to success is only needing to refer to it until it is time to renew. This can only be accomplished by doing the upfront work required to ensure this will be a strong relationship.
Prosser recalls, "They weren't coming to the table with innovative things; they weren't excited about the technology. They were jaded and cocky; they thought they had the contract in the bag."
Faced with all this, Prosser made his decision. And instead of getting a contract twice as big as the original one, the outsourcer lost the contract completely.
To gain those cost savings he had been looking for, Prosser took management of the county's IT resources in-house and hired away from the outsourcer several IT people whose work he was happy with.
Harry Turnbull: Just like car insurance every once in a while you need to shop around to find the best fit. Some large companies act like they have a monopoly and their arrogance gets in the way of what is best for us. When shopping around, keep in mind that doing work in-house is not a bad word, sometimes it makes the most sense.
The outsourcer "told us we wouldn't be able to [manage IT resources in-house] and save money, but we did," Prosser says.
But the story doesn't end there. After leaving the county of his own accord in 2008 to start LifeBridge International, a service firm that designs travel programs for personal enrichment where he is now CTO, Prosser heard through back channels that executives at the outsourcer claimed that they had gotten him fired.
That kind of gossip, he says, was inappropriate. More important was the question of the company's competence. "I don't care who you are," Prosser says. "What matters is, can you get the job done?"
Vendor visibility obscured by the cloud
In this era of cloud computing, it can be tricky for IT buyers to figure out who they're really purchasing services from, and therefore who is ultimately responsible when things go wrong. The vendor from which customers buy a cloud service -- such as storage or email -- often is the customer of yet another cloud service supplying infrastructure for that service. Given all that, determining the cause of a problem can be difficult and frustrating for IT departments.
Harry Turnbull: Cloud is just a buzzword for somebody else has our stuff! As was stated earlier, IT projects are complex, moving some things into the cloud simply adds another (or different) level of complexity to consider and provides even more justification for doing a very thorough job up front to know what you are getting into. If a cloud relationship goes bad, it could present a new set of challenges to undo.
About a year ago, a cloud storage provider suggested to StenTel Transcription and Catuogno Court Reporting that the company buy its services both for internal use and to resell to Catuogno's reporting and transcription customers.
Catuogno already had some experience providing technology services to its customer base with an encrypted email offering and thought adding storage services would create a new line of business, says CIO Blake Martin.
"The demos looked slick, the vendor made great promises, and we thought we had done our due diligence," says Martin. "After deploying, we found out that the service was a lot less mature than we had thought. We had a lot of issues getting it properly configured. We felt a bit abandoned."
What's more, during the sales process, the cloud storage provider misrepresented the depth of its relationship with a second company, the one that actually developed the core technology behind the service. When Catuogno ran into trouble, the cloud company didn't have enough clout with the core vendor to bring about an effective resolution.
"It's not that they were unresponsive. It's that they couldn't address the problems themselves and they didn't have enough influence" with the core technology developer, Martin says.
Catuogno stayed with the service for about seven months before giving up, and the company is now regrouping to decide whether it wants to try using and reselling another cloud vendor's storage services.
The incident taught Martin to get a clear understanding upfront of what the vendor is providing itself and what its partners are bringing to the table. "The lesson we learned is to really understand and explore the relationship between who you're buying from and who provides what that vendor is selling," he says.
Sometimes the buyer is to blame
To be fair, it's not always exaggeration or half-truths on the part of the vendor that leads to buyer disappointment; sometimes it's due to a lack of education, research or comparative shopping on the customer's end.
A.G. von Luternow, a Georgia-based tech consultant, says he's seen clients in a hurry to make a purchase wind up with a bad case of buyer's remorse.
Harry Turnbull: It is absolutely our responsibility to ensure that we (including our client) are choosing the right solution for a given issue. We do that through up front work, understanding it is the vendor’s job to sell and ours to scrutinize, creating the relationships with our internal customers and ensuring we are working together to find the right solution.
One retailing client wanted to purchase cataloging software but evaluated only one tool. The company decided to go with it without first checking with von Luternow.
The retailer ended up having to make some significant hardware investments to be able to run the software. It also wanted to integrate the new software with desktop applications, which the product didn't support.
"They chose the wrong tool because they only looked at one, and now it really doesn't work the way they wanted it to," says von Luternow. "The vendor didn't misrepresent the product -- it does everything they said it would. The issue is that the people who were choosing the tool lost track of what problem they were trying to solve. And now they are stuck with it."
Harry Turnbull: A vendor contract is a relationship and just like any other, it requires the appropriate fostering and nurturing. If you have to resort to the Terms and Conditions of the contract it is more than likely already too late, better to take the time up front to ensure the relationship will stay strong.