Companies today are under increasing pressure to make better business decisions in less time, with less risk, while producing higher quality results. Their challenges are enormous, as are the many issues that can arise and potentially jeopardize their success.
One of the most pervasive problems companies face is the consistently poor quality of the internal data that they use to draw conclusions and make decisions. Poor data quality is not a new problem, but now solving it is easier than before because companies no longer have to rely on methods that require them to ‘boil the ocean’. Streamlined approaches to data governance that incorporate new processes and data stewardship technologies enable more agile methods for improving data quality.
The problem with data governance programs thus far has been that most companies were taking a top-down approach, while more pressing short-term business demands were derailing efforts and distracting resources. Executives responsible for these programs and line-of-business managers, whose cooperation was needed to make the processes successful, were always scrambling to make their own revenue numbers, launch new products and meet other required business objectives, rather than focusing on the often laborious data governance process, which only produced intangible results.
As a result, today only a very small percentage of companies actually have an active data governance program in place because, in most cases, a top-down approach just doesn’t work. In a typical top-down waterfall approach to data governance, a company spends six months forming a committee, another six months defining and stating the problems, another six months gathering requirements, and another six months arguing about terminology. By the end of a two-year process, a bunch of people have wasted their time in endless meetings with little to show for it.
A BETTER APPROACH
An alternative to top-down data governance, and a much better way to address the problem, is for companies to adopt a more agile approach. By introducing more agile processes, companies can achieve quick wins by implementing data governance processes and policies in smaller pieces, learning from and adapting the approach with each segment, taking time off in between increments to focus on pressing business goals, and then coming back together to address another data domain, while making steady progress over time.
For example, with an agile approach, a company could decide to focus its efforts on a master data management project for its customer data that it could actually solve within six to eight months, which is quite different from a top-down approach that takes years just to get started!
Companies that want to try an agile data governance approach should follow a number of guidelines that will assist them in a successful implementation. Before we get into the specific details of an agile process, let’s take a step back and consider some of the broader issues that are key to the success of any project.
First and foremost, determining up front which data fixes are going to deliver the greatest business benefits and focusing on fixing those first will help guarantee that the project is a success.
Second, limiting the size of the data governance team, and allowing the team to evolve as different data is being addressed at each stage of the project will help streamline the process and eliminate many of the potential political issues.
Finally, drafting a solid team of data stewards for each piece of the project will help to ensure success.
Before the overall project begins a company needs to select a small, core data governance board made up of executives who can authoritatively represent the business goals of the entire organization. The board needs to have enough visibility to be able to sort out the biggest data problems the enterprise faces and determine which critical few problems should be tackled first, including the very first project, which should deliver the biggest overall bang for the buck.
Data governance problems that should be addressed are those where poor data quality have the most direct impact on bottom line profitability, productivity, cycle times, customer satisfaction, risk, reputation, cost savings, employee morale, etc. The data governance board will ultimately be responsible for overseeing the entire data governance process, but will not manage individual pieces of the project. Beware of the common practice of assembling larger groups for data governance projects. The thinking is that larger groups help ensure that everyone has a voice, but in reality this can seriously compromise agility and drag out a project forever.
STEPS TO SUCCESS
Each iteration of an agile data governance project will be managed by an implementation team which will work with others within the organization to follow a number of specific steps that will result in successful completion. Below are the recommended steps companies should follow for success.
1 Select the data governance implementation team for the first project. Once data priorities have been set, the core executive board should select the team that will manage the first piece of the project. Executives who will all benefit from fixing a specific type of data are most likely to work well on the team and ensure that a project gets completed in a timely fashion. Each piece of the project will require different team members, depending on which data area is being addressed. For example, if a company is resolving B2B hierarchy data, that problem will require a different group of executives than if product catalogue data is being addressed.
2 Define the data problem. The data governance implementation team should evaluate and determine the size and scope of the data problem they are going to address. This team needs to clearly define the business problems they are trying to solve or improve, and then ruthlessly manage the process to ensure that everyone stays focused on that one data domain and worries about the rest later. If the project for a single domain of data takes more than eight months to complete, chances are the scope is too big.
3 Draft the data steward team. The data governance team should create a team of data stewards made up of those individuals with the most knowledge of the data being addressed in each piece of the project. Data stewards do most of the hard work with the specific data and are responsible for acting on behalf of data owners to employ technologies to find poor quality data, remediate and repair data, and prevent errors in new data. In a global corporation, it would not be unusual to have dozens of data stewards spread across the world with the common task of solving a specific data problem. The data governance board should provide ‘air cover’ to free data stewards up for this task, and set aggressive deadlines to avoid analysis paralysis.
4 Validate or disprove assumptions of the data governance team. The first job of the data steward team will be to investigate data quality problems in the first chosen data domain. This involves data discovery work to understand the data quality issues, profiling data across system boundaries, looking at messages that carry the data payload, and examining ETL transformations and data integration adaptors and connectors. The job of the data stewards is to determine the extent of data quality issues, root causes of data problems, and the extent of potential damage when conducting a proof of concept on the data. In very large enterprises, for scope management reasons, it may also be wise to begin by focusing on a subset of the global systems.
5 Establish policies for making changes to data. When the data steward team delivers the results of their analyses, the data governance team needs to step back in to create new policies for data and to secure board approval for their recommendations. These should be business policies, not technology policies, which enforce how data is handled within and between participating systems. They should define policies for system-independent data as well as transaction-specific data quality, and for addressing and solving often contentious cross-organizational data sharing. When participating systems have unique data integrity constraints that do not represent shared business rules, the governance teams should ensure that these constraints do not leak into business policies.
6 Enlist internal IT groups required to fix problems. The data governance and steward teams will work together to kick off the actual project implementation. This phase involves calling on enterprise architects from internal IT groups to help design the best solution before specific problems are identified and fixes are made. Too often, repairs are made ‘quick and dirty’, which can cause additional data quality problems now or later. It is critical to specify metrics reporting as part of the project design, so that exceptions and error rates are captured and reported as errors occur. Inevitably the data governance team and board, as well as data stewards, will be called upon to clarify business rules.
7 Compare results, evaluate, and determine next domain. After testing the solution, bringing data into compliance across all participating systems, and implementing new data quality tools, the team should compare the results and exceptions with the new business policies to determine where issues still exist (it may take several iterations to cure a single data domain completely). At this point, evaluate what worked in the overall process (and what didn’t) and determine which data domain to tackle next.
The key to changing the current mindset that data governance processes are long and tiresome and yield very few results is to focus teams on achieving quick wins using a lighter weight, more agile approach, such as the process outlined above. For companies that want to make better business decisions by leveraging higher quality data, what is most critical is to begin implementing an agile data governance process immediately. Within six to eight months, these organizations will begin reaping the rewards. New approaches to data governance do not require executives to boil the ocean. With a few simple, easily definable steps, a C-level executive tasked with managing enterprise-wide data governance can begin the process and be successful.
Remember, it’s better to minimize scope and ensure success, especially for busy executives, than to invite a cast of thousands to be part of a never-ending process. Ultimately, an agile data governance approach will result in faster, measurable results to solving an organization’s pervasive data quality issues. A company can then build on its successes by iterating through the agile data governance process.
Marty Moseley is a speaker and author on technology topics including data governance, customer data integration, master data management, and software architecture. Moseley is the chief technology officer at Initiate Systems, a provider of master data management software. He can be reached at mmoseley@InitiateSystems.com.