August 12, 2011 | By Rey Villar
You made a business case for building a Data Governance organization to solve your Business Intelligence challenges, got your funding, and hired a couple of seasoned consultants to help design the governance strategy, shaped a DG council and stewardship organization, and transcribed policies and procedures to put DG in motion. A couple of years have gone by and you are still facing those key challenges. What happened?
Is your data governance program working? How do you know? Are the sponsors happy? Are the analysts working more effectively? How do you know when the effort has paid big dividends? As Business Intelligence practitioners, we attach metrics to all sorts of activities as best practices. Shouldn’t capture of success metrics at program inception and on an ongoing basis be a best practice for data governance as well?
Many organizations don’t consider the need for metrics to measure the long-term effectiveness of their Data Governance program. Don’t feel badly – if you haven’t quantified DG success, you’re not alone. Companies in all industries struggle to capture relevant and meaningful DG metrics. Thankfully, there is no shortage of data that can help you to measure the impact and effectiveness of your Data Governance efforts. The key is to use Data Governance Metrics to measure and demonstrate program effectiveness.
What is Data Governance Metrics?
Data Governance Metrics is foundational to measuring the success and effectiveness of a Data Governance program. Consider the set of metrics designed to measure the effectiveness of the Data Governance. Like other business performance measures, metrics should be managed and tracked at all levels.
What defines a good Data Governance Metric?
Prior to defining the DG metrics, you should understand the key characteristics of a reasonable DG metric and then explore how to map those characteristics to the measurable aspects of data governance. Metrics should be specific, measurable, attainable, realistic and timely. The following list of questions provides some guidance to jump-start the approach:
- Specific – Identify set of specific metrics that will measure the success of the DG program
- Measureable – Clearly defined, simple to understand and easy to measure
- Actionable – easy to capture, realistic and practical, and quantifiable.
- Realistic – Does the metric have business relevance?—i.e. defined within a business context that explains how the metric score correlates to improved business performance
- Timely – A timely goal is intended to establish a sense of urgency and measure over a period of time to analyze the trend.
The acid test for each metric is whether it is clearly defined, capable of measurement, and directly relevant to improving program effectiveness. The Data Governance Council will be a good judge, and they should review and approve all metrics. Metrics are created by either DG Councils or data stewards with input from data analysts.
Key Data Governance Metrics
Data Governance metrics should be identified and tracked prior to implementing the Data Governance program to baseline performance. You will want to capture these metrics periodically and store in a table or database to review progress with the metrics over time.
Commonly measured dimensions of Data Governance include completeness, consistency, timeliness, and uniqueness, although the range of possible dimensions is limited only by the ability to provide a method for measurement. Metrics can be composed of directly measured rules or more complex metrics that are defined as weighted averages of collected scores. Metrics from a business case or ROI for the DG program proposal can provide a starting point.
At a high level there are two broad categories of DG Metrics:
- Quantitative Metrics – Data centric Metrics that can be measured as hard benefits like savings in manpower or operational cost savings, etc.
- Qualitative Metrics – Metrics that measure soft benefits like improved customer satisfaction survey results, increased industry standard data quality scores, etc.
Listed below are some sample Quantitative Metrics I have used in the past to measure the program effectiveness for healthcare clients.
In my next blog, I’ll explore indirect metrics as well as how to design and implement Data Governance Metrics as the foundational building blocks for a successful Data Governance program. If you’ve already walked this path and have some learning to share, let me hear from you. Stay tuned.