Best practices in enterprise integration

IT World Canada is pleased to offer research from Meta Group Inc. This content will be featured on for three months from the article’s publication date.

Integration is critical to the main initiatives that CIOs will face in 2004/05, regardless of the focus on risk management, regulation compliance, M&A, cost management, or outsourcing. We expect the demand for integration to continue and even grow. As revealed in META Group’s adaptive organization multiclient survey, corporate CIOs understand that integration supports the adaptive organization initiative and helps organizations to improve business processes and practices. In this context, it is important to understand how organizations have run their existing integration projects and to describe the best practices they use to enforce the success of the initiatives.

META Trend: Through 2004/05, the enterprise application integration (EAI) market will grow, though limited-scope projects will remain the norm. Requirements will continue to shift from standalone integration projects to consolidated software frameworks that enable composition of internally and externally facing service-oriented applications. By 2006/07, users will routinely evaluate integration frameworks based on their participation in heterogeneous service-oriented architectures.

META Trend: Data management will increasingly be viewed as a part of overall integration architectures and designs (2004/05). This will cause organizations to consolidate these efforts (i.e., center of excellence for integration services). Enterprise information integration, ETL, and enterprise application integration will ultimately disappear as distinct categories (2008/09), surviving only as various subsets of capabilities in the service-oriented architecture (2009).

Our client inquiries show that, for the past three years, organizations are reluctant to start enterprisewide strategic integration initiatives. Instead, they attempt to solve limited and tactical problems. During our “Case Studies in Enterprise Integration” teleconference, one client wanted to modernize his application portfolio by injecting new technologies into a set of multiple, disparate, and incompatible existing applications without retiring anything. Another client started a project to integrate all his distributed legacy applications into a new centralized ERP solution and therefore rationalized the IT budget (see Teleconference 2090). Organizations deploy pragmatic solutions to business integration requirements with a strategic vision to enforce adaptability. Three best practices can be highlighted as critical success factors of integration projects.

Integration Governance Model. To be successful, integration projects need to be executed with a level of consistency and governance to address some of the key challenges. For example, lack of confidence in the correctness of information that is being delivered to the organization leads to poor and delayed decision making. Not having interface design principles and common formats for business object definition drives to point-to-point application approaches and prevents reuse of already defined interfaces.

Organizations need to develop a governance model so that they can have consistency from every perspective (e.g., organization, architecture, development, implementation):

- The model describes a funding process that ensures ongoing investment for integration. Many requirements will rise during the project, and changes need to be taken into account. The model must also compensate for the “first man in” problem, where the initial developer of a reusable asset bears all the costs.

- The model clearly defines the roles and the responsibilities of all the participants in the ongoing integration initiative: business users, business analysts, technical designers, developers, and architects. It provides negotiation mechanisms and service-level agreements between organizations to define a contract where ownership, definition of the interlocking processes, and change procedures are clearly defined.

- The model defines a methodology to document technical items such as record definitions, structures, interfaces, and flows. It is managed in a shared repository, and templates describe the items.

- Governance also addresses support and help-desk issues, because clear support definitions are required when applications are in production.

This governance model enables the various disparate individuals and interests that need to be involved in any integration process to reach an agreement. Integrations are agreements between not only applications, but also the business organizations that such applications support. In many cases, there may be no existing mechanisms for the business organizations to come to agreement, and these must be developed to successfully create reusable integrations.

Centre of Excellence (COE) for Integration Services. Because enterprise application integration is a critical real-time system, it has to be managed centrally as do all other critical resources (e.g., networks, databases, application servers) and not individually as part of one application. To support large and complex multiprocess scopes spanning multiple domains, successful organizations have established a COE for integration services to improve knowledge sharing and cross-application consistency:

- Specifically funded, the COE for integration services is staffed with representatives from all parts of the organization (e.g., management, line of business, IT). Its members enforce the integration governance model. This team of seasoned people educates the organization on service-oriented architecture and loose-coupling concepts, and it trains the working staff on critical business and IT-specific integration skills.

- The COE for integration services is responsible for enforcing integration architectural standards and for helping project teams to decide on the best architectural design. The COE defines design patterns and intermediate document formats. The COE also manages the repository.

- The COE for integration services helps project teams in developing the integration layer to their applications (e.g., analysis and design, code, build-and-unit test, implementation).

- The COE for integration services helps operation teams in supporting all operational aspects (e.g., administration, optimization, service-level agreements, support).

A COE for integration services enables consistency of planning, architecture, development, and operation between all projects and enables reusing existing skills.

Reuse. Although few organizations have set up a formal program to encourage and enforce reuse, it has been a critical factor of successful integration projects. Reuse currently exists within limited domains (e.g., within project teams), but new processes and roles are required for enterprise reuse. It requires investment in process, personnel, and technology to create reusable services that have a life cycle independent of the application that creates or consumes the services:

- Reuse is not a technical problem, but its barriers are trust issues within the organization. For example, people may be questioning the validity of the information warehouse because confidence in the data is eroded. The fundamental challenges around reuse are often human communication challenges. Apart from creating reusable items, an organization needs to communicate its availability and its characteristics. On the other side, people must be encouraged as part of their daily work practices to reuse such items.

- Another aspect of reuse is the knowledge and the expertise that exists in the organization. Members of the COE for integration services need to share their knowledge and skills with development teams and business analysts.

- The ability to reuse development patterns dramatically cuts down the time to build integration on new projects.

- Intermediate document formats reduce the number of many-to-many application formats. It also reduces semantics transformations between applications (e.g., a purchase-order format can be defined across all applications). It requires public grammar with agreed semantics.

Technology differences and business pressures create a chance for the success of a reuse program. However, any successful reuse program must focus on changing behaviour and measurements, rather than on technology characteristics of reused entities.

Business Impact: Enterprise integration best practices will improve the productivity of the IT organization.

Bottom Line: To ensure success of integration projects, organizations should focus on governance strategy, build a COE for integration services, and promote reuse.