The Nine Circles of Data Quality Hell

May 27, 2009
148 Views

 “Abandon all hope, ye who enter here.” 

In Dante’s Inferno, these words are inscribed above the entrance into hell.  The Roman poet Virgil was Dante’s guide through its nine circles, each an allegory for unrepentant sins beyond forgiveness.

The Very Model of a Modern DQ General will be your guide on this journey through nine of the most common mistakes that can doom your data quality project:

1. Thinking data quality is an IT issue (or a business issue) Data quality is not an IT issue.  Data quality is also not a business issue.  Data quality is everyone’s issue.  Successful data quality projects are driven by an executive management mandate for the business and IT to forge an ongoing and iterative collaboration throughout the entire project.  The business usually owns the data and understands its meaning and use in the day to day operation of the enterprise and must partner with IT in defining the necessary data quality standards and processes.

2. Waiting for poor data quality to affect you Data quality projects ar

 “Abandon all hope, ye who enter here.” 

In Dante’s Inferno, these words are inscribed above the entrance into hell.  The Roman poet Virgil was Dante’s guide through its nine circles, each an allegory for unrepentant sins beyond forgiveness.

The Very Model of a Modern DQ General will be your guide on this journey through nine of the most common mistakes that can doom your data quality project:

1. Thinking data quality is an IT issue (or a business issue) Data quality is not an IT issue.  Data quality is also not a business issue.  Data quality is everyone’s issue.  Successful data quality projects are driven by an executive management mandate for the business and IT to forge an ongoing and iterative collaboration throughout the entire project.  The business usually owns the data and understands its meaning and use in the day to day operation of the enterprise and must partner with IT in defining the necessary data quality standards and processes.

2. Waiting for poor data quality to affect you Data quality projects are often launched in the aftermath of an event when poor data quality negatively impacted decision-critical enterprise information.  Some examples include a customer service nightmare, a regulatory compliance failure or a financial reporting scandal.  Whatever the triggering event, a common response is data quality suddenly becomes prioritized as a critical issue.

3. Believing technology alone is the solution Although incredible advancements continue, technology alone cannot provide the solution.  Data quality requires a holistic approach involving people, process and technology.  Your project can only be successful when people take on the challenge united by collaboration, guided by an effective methodology, and of course, implemented with amazing technology.

4. Listening only to the expert An expert can be an invaluable member of the data quality project team.  However, sometimes an expert can dominate the decision making process.  The expert’s perspective needs to be combined with the diversity of the entire project team in order for success to be possible.

5. Losing focus on the data The complexity of your data quality project can sometimes work against your best intentions.  It is easy to get pulled into the mechanics of documenting the business requirements and functional specifications and then charging ahead with application development.  Once the project achieves some momentum, it can take on a life of its own and the focus becomes more and more about making progress against the tasks in the project plan, and less and less on the project’s actual goal, which is to improve the quality of your data.

  • This common mistake was the theme of my post: Data Gazers.

6. Chasing perfection An obsessive-compulsive quest to find and fix every data quality problem is a laudable pursuit but ultimately a self-defeating cause.  Data quality problems can be very insidious and even the best data quality process will still produce exceptions.  Although this is easy to accept in theory, it is notoriously difficult to accept in practice.  Do not let the pursuit of perfection undermine your data quality project.

7. Viewing your data quality assessment as a one-time event Your data quality project should begin with a data quality assessment to assist with aligning perception with reality and to get the project off to a good start by providing a clear direction and a working definition of success.  However, the data quality assessment is not a one-time event that ends when development begins.  You should perform iterative data quality assessments throughout the entire development lifecycle.

8. Forgetting about the people People, process and technology.  All three are necessary for success on your data quality project.  However, I have found that the easiest one to forget about (and by far the most important of the three) is people.

9. Assuming if you build it, data quality will come There are many important considerations when planning a data quality project.  One of the most important is to realize that data quality problems cannot be permanently “fixed” by implementing a one-time “solution” that doesn’t require ongoing improvements.

Knowing these common mistakes is no guarantee that your data quality project couldn’t still find itself lost in a dark wood.

However, knowledge could help you realize when you have strayed from the right road and light a path to find your way back.

Link to original post