SOA may have seemed the savior of bad software architecture and poor development project planning, but the reality is that it's a complex and difficult venture. Thus, the number of failed SOA projects is about equal to the successful ones. In other words, you have a 50 per cent chance of failing, and the odds of failure are even greater if you work within a larger Global 2000 organization or within the government.
But key patterns are emerging from the successful SOA efforts, patterns that can help you determine whether your SOA is a failure or a success.
The most important lesson from these patterns is that SOA is as much about old-school IT disciplines as it is about new, inventive technology. Moreover, it's about changing an organization from the people down to the technology, driving a systemic and valuable change. The patterns of success likewise follow that change from the people on down to the technology.
People: From leadership to staff, the right aptitude and commitment is critical
The fundamental cause of SOA failure is the lack of good people working the architecture, from leadership to staff. This lack is not of numbers, but of knowledge and the vision to drive change.
People-based SOA failure begins at the top. A recent Burton Group study determined that the arrival of a new CIO typically spelled success for SOA. In essence, innovative leadership and the ability to change the culture and then the architecture is a clear critical success factor.
Moreover, critical to SOA success is the presence of a leadership team that values an investment in infrastructure and understands the long-term value of being more agile and efficient, and, better yet, is willing to make the investment. SOA is not cheap. Indeed, it's a huge systemic change in how you build, deploy, design, test, and architect your information technology. It's an investment that goes well beyond the millions of dollars required to acquire the necessary training, consulting, and then the technology required to make the changes.
The SOA investment cannot be treated as a one-time "transformation" project. Instead, you need to consider SOA as a journey, not a project, and -- for the purposes of budgeting -- treat it as a series of projects that make up the journey.
The SOA effort does need its value clearly defined, and the investment and commitment to achieve that value made up front.
Because SOA is really about a journey, not a project, enterprises need to take a long-term view of SOA. It's a multi-year and typically multi-million-dollar commitment to drive a systemic change to the core IT mechanism. But most SOA "projects" are stopped at some point, typically due to budget issues or to the need to redirect resources at some tactical short-term need. Thus, the SOA never had a chance; there was no follow-through. Executive and IT management must not allow this.
SOA also involves two fundamental business changes that IT historically doesn't have the clout to make happen: One is the sharing of processes across political fiefdoms that fear loss of control. The other is that SOA requires a rethinking of fundamental processes, which is not only hard but also challenges established practices, resource allocations, political power, and so on.
At the management and staff levels, the people requirements for successful SOA are even more fundamental. Although many would love to have the existing team drive them to the new world of SOA, the harsh truth is that many in that team won't have the skills or the aptitude. You need to make some hard decisions up front as to who will take you forward. This means you must replace people or you must augment staff. Either approach is costly.
Many companies employ mentors or consultants to help learn the new ways of SOA. Others invest in SOA training, or even bring in entirely new teams as either consultants or employees to assist in "the change."
No matter what you do, never, ever try to drive SOA with the wrong people; it just won't work.
Processes: SOA requires a change in how you develop, manage, and test applications
Approaching SOA means changing how you approach architecture and systems development. In the past, many companies have approached systems by dragging whatever seemed cool at the time into the enterprise to solve a tactical problem. Of course, tactical problems lead to other tactical problems, and these companies kept digging a deeper hole in terms of architectural complexity and efficiency.
Thus, you need to approach SOA using a well-thought-out, practical process that lets you break the architectural domain down to a functional primitive, then build it back up again as a SOA. Unless you're the smallest of enterprises, you also need to break these domains into achievable chunks that can be worked on in sequence, or later in parallel.
Then it's time to do some real work, including looking at the information within the problem domain, meaning application semantics and metadata. This takes a ton of work, because most enterprises typically don't have a good semantic-level understanding of their enterprise systems. Thus, this effort is usually not a process of reviewing common artifacts, but of creating new ones. Many SOA efforts skip this step, which cripples what they end up delivering.
From the information step, you move to the services: defining existing and new services that will make up your SOA. This is all about figuring out what you have, determining what you need, and then normalizing the services into a usable and well-defined grid. You need to make sure you define and document the services, and then link them back to the metadata model you've created.
Keep in mind that most of your services will be existing services, externalized through some sort of middleware mechanism. Most people think SOA is about building all-new services, but that's almost never the case. Many SOA efforts fail because they become excuses to reinvent the wheel, not solve pressing business needs.
You then need to define and create processes that bind the services together into business solutions. There are three approaches here you can use. First, you can create service-oriented business applications, a way to programmatically bind services together into processes or applications. Second, you can leverage an orchestration layer, such BPEL, to bind services together to form business solutions. Third, you can leverage traditional process integration tools to bind services into solutions. No matter what approach you use, you need to make sure to include requirements as well as design.
Of course, there are other process things to do, including the creation of test plans and selection of testing tools for the services. Also, you need to create a SOA governance strategy, determine the governance process based on that strategy, and select tools to support the creation and enforcement of governance policies.
Architecture: The core of SOA is usually given short shrift
The "A" in SOA is architecture. Most people forget that, opting instead to jump right into the technology. Why? Because architecture requires a huge effort and is boring, whereas playing with cool technology is fun and fast. Unfortunately, you really have to spend the time to plan and lead the enterprise in the right directions. Create a strategy, processes, architecture, links to business realities, budgets, and then -- the most important action -- lead the technology teams in the proper directions.
Although architecture is fundamental to SOA, it is often neglected in so-called SOA efforts. That's because most enterprise architects within the larger enterprises don't have the authority and reach t