Several recent articles have discussed why many SOA initiatives are failing. In early July, when vice President and research director Anne Thomas Manes presented at the Burton Group’s annual Catalyst conference, she said most SOA failures are due to people and cultural issues more often than for technology issues. I totally agree with her assessment, as I have been blogging about this same issue for a long time.
So now we know who to blame for failed SOA initiatives. It’s the people, stupid! But just why do people make SOA fail? Let me count the ways
1. They fail to explain SOA’s business value.
One of the most common mistakes IT people make is that they approach SOA purely from a technology perspective. They spent a great deal of time and effort on architecture, governance and vendor assessments (which is good), but they forget that SOA needs to solve real business problems. So they spend a huge amount of time and money building out the architecture—only to find that when they are done, nobody in the business understands the benefits and are not interested in the technology.
Recommendation: Start with real business problems first. This is why BPM (business process management) is the “killer app” for SOA. BPM solves several business problems by improving and automating business processes. It provides visibility into operational performance, enhances agility by allowing the business to change their processes dynamically without IT involvement, eliminates waste—thus reducing costs—and much more. Start by showing the business how SOA will solve real business problems first. Then address the technology issues.
2. They underestimate the impact of organizational change.
As with any transformational initiative, resistance to change is a project killer. SOA brings massive amounts of change to an organization, especially if the organization does not have a well established enterprise architecture in place. Fear of the unknown is the greatest contributor of resistance to change. People need to understand WIIFM (what’s in it for me) and why changing their ways is good for both them and the company. The challenge is people at different levels within the organization are affected in different ways. Each level of the business has concerns which need to be addressed which must be solved at an individual basis.
Recommendation: Create an organizational change management (OCM) plan. I would go one step further and hire an external OCM expert to help the leadership team of the SOA initiative deal with change. I am a big fan of John Kotter’s eight-step methodology.
3. They fail to secure strong executive sponsorship.
Without strong executive sponsorship, it is highly unlikely that your SOA initiative will accomplish its goals. SOA spans multiple departments, multiple systems, and is a major undertaking. You need a strong executive with clout to keep the initiative moving forward and break down the barriers along the way. But clout by itself is not enough. This person also needs to have enough time to focus on the SOA initiative to keep the sense of urgency at a high level.
Recommendation: If your SOA is aligned with key business drivers, the executive sponsor should be a high ranking business person who benefits substantially from the implementation. Let the business own and drive the portfolio of projects that drive the SOA road map. In technology companies, it is highly likely that the CEO, CIO, CTO or Chief Architect is the executive sponsor. Whoever you chose, this person must be able to move mountains and should have a proven track record of successful leadership.
4. They attempt to do SOA “on the cheap.”
SOA is not something you buy, it is something you do. Some companies attempt SOA with limited budgets. In addition to all of the middleware that is required, there are huge investments in governance tools, training, consulting, infrastructure and security.
Managing SOA in a production environment is challenging because of its distributed and loosely coupled nature. Don’t skimp on the lifecycle management tools or troubleshooting will be like trying to find a needle in a haystack. Some companies will try to take on SOA projects without any outside help to shave off the high cost of consultants. Unless you are loaded with people experienced in SOA, trying this without outside help in order to save money can be a recipe for disaster.
Recommendation: Create an SOA road map with a portfolio of projects and a vision of the long term benefits that SOA will bring to the company. Create the financial justification for the entire SOA initiative and show the ROI, NPV, IRR or whatever the most important financial indicators are for the company. If you build a good enough business case, there should be enough money to fund the initiative. Also, several great open source products are available that can greatly reduce the overall costs of your SOA implementation.
5. They lack the required skills to deliver SOA.
There are many specialized roles and skill sets required which probably do not exist within the organization. You need SOA architects, business process modelers, administrators of the tools within the stack, data architects and many other skills. These positions are not cheap. Trying to implement SOA without any SOA experience is a major mistake. SOA affects all IT departments including testing, infrastructure and security. This is much more complex then sending a few developers to a few training classes. Don’t forget the business, either. The business needs training on process improvement and possibly even on the BPM tools as well.
Recommendation: Develop an extensive training and resource plan and include it as part of the initial request for funds when delivering the business case for SOA. Try to reduce the number of times you need to ask for money and get as much as you can up front. Otherwise, management may view the SOA initiative as an endless drain on capital.
6. They have poor project management.
At the end of the day, it still boils down to a company’s ability to manage projects. Project managers must manage scope, mitigate risks, keep everyone on schedule and provide the proper communications to people at all levels. Requirements gathering is critical and analysis paralysis must be avoided. If your organization struggles delivering normal projects, your chances of success with SOA will be twice as challenging.
Recommendation: Put your best project management resource(s) on this project. Or go out and get a rock star or two to help lead this initiative. Whoever you chose should have a track record for delivering huge, transformational initiatives. To make matters even more challenging, this person needs to be technical enough to understand SOA from a conceptual level.
7. They think of SOA as a project instead of an architecture.
Many companies are naive and think that implementing SOA is just another project. SOA is a software architecture and only achieves desired benefits when a company adheres to the core principals of service-orientation and ensures that deliverables are consistent with the architectural vision and road map. SOA requires specialization. A business service may be built through the efforts of an SOA architect, a developer, a data architect, a network architect, and a security specialist. Gone are the days where one person wears all the hats. There is also specialization within the layers of the stack. You have user interface designers, business process models, data services experts, business rules specialists, ESB experts, etc. All of these specialists may be working on the same services concurrently which requires high levels of collaboration.
Recommendation: The standard IT team structure is not effective for SOA. Think outside the box. I prefer a matrix organization and a collaborative war room environment. Tear down the cubes and set up an open space to allow for these specialists to work closely together. It also helps to have the business and testers in the room also. Put up white boards everywhere. Eliminate as many schedule meetings as possible and choose more collaborative techniques instead.
8. They underestimate the complexity of SOA.
You don’t know what you don’t know. Conceptually, SOA is simply the next evolution of what IT has been building over the years. It is not a hard concept to understand, but it is hard to implement correctly. The beauty of SOA and BPM is the simplicity it brings to the end users by integrating various back end systems so they look like one composite application to the user. The downside to SOA is that this greatly increases the complexity of building and managing software. Building SOA is a software engineering exercise. This is not drag-n-drop development effort and many developers will struggle to make the transition. SOA requires adherence to standards and best practices (governance) and needs talented individuals who understand complex concepts to be able to deliver.
There is so much to do to implement SOA that often security is an afterthought. It is critical that security requirements are gathered early so that the underlying architecture supports security from its inception. Otherwise, it is highly likely that major changes in architecture will need to occur if security is addressed later on.
Recommendation: No matter how conservative you are, expect to run into various technical road blocks along the way. Build in time as you will run into various integration issues, some caused by your code and some caused by the tools themselves. The vendor products are all far from being mature and there will be issues. Set realistic expectations and don’t try to take on too much too soon. Start small, deliver value often, and build on it. Build security in from the inception; don’t let it be an afterthought.
9. They fail to implement and adhere to SOA governance.
Governance is a dirty word to many people. Anything that sounds remotely close to government can’t be good, right? Wrong! Call it SOA management and maybe people won’t shiver as much.
Regardless, to achieve the benefits of SOA (reuse, flexibility, agility), the team must adhere to the architectural guidelines that the company adopts. This is what is called design-time governance. Without design-time governance you will likely wind up with JABOWS (just a bunch of Web services). If that occurs, you can throw the ROI out the window because you wind up building everything from scratch each time. When done right, SOA becomes more cost effective over time. Eventually the development effort will shift from building services to consuming services. Jason Bloomberg of Zapthink calls this the tipping point, when the SOA starts reaping the benefits of agility and flexibility.
Then there is run-time governance. This is where you proactively manage the health of your production SOA environment. Run-time governance allows you to see what services are being consumed, enforce policies and SLAs, troubleshoot, analyze performance and manage all assets. Don’t think that once you deploy that you are done. Managing a distributed environment is not a task to be taken lightly.
Recommendation: Treat governance as a fully funded initiative that runs alongside of your SOA implementation. There should be a dedicated team (which usually lives within Enterprise Architecture) with its own road map and long term vision. Don’t try to implement governance overnight. This is a journey; it will take several years to reach a high level of maturity. As your governance matures, so does your SOA. Invest in a registry, repository and service management tools. You also need new testing tools to test governance.
10. They let the vendors drive the architecture.
Ron Schmelzer at Zapthink coined the term Vendor-Driven Architecture (VDA). Relying too much on vendors can be a disaster. The vendors’ goal is to sell you as much stuff as possible. Your goal is to implement SOA successfully and to provide your company with maximum benefits at minimal cost. Do you see the conflict of interest?
Also, the vendors promise flawless integration if you purchase all of your tools within their stack. The reality is, they have purchased so many products from other companies that their stacks do not deliver any better integration than if you bought the tools from a variety of vendors.
Recommendation: Figure out what you need before you talk to the vendors. Perform a very thorough vendor evaluation process. When you narrow it down to a few vendors, have them come on site to perform a proof of concept for which you provide the requirements. Watch them execute on it before your very own eyes. This is where the vendors can no longer hide behind pretty PowerPoint slides; it can prevent you from making a colossal mistake. Do your homework. Read blogs from practitioners, talk to consulting firms who use the tools, talk to other companies that have implemented SOA and talk to vendor references. Do not take any short cuts; you will have to live with the decisions you make.
