Getting to the Root Cause of Data Quality Issues.

As a marketing manager, my primary responsibility is to build marketing campaigns that make people want to use our products.  To do this I need to have solid data to work from, such as contact information, needs, and levels of interest about our prospects and clients. For this reason, my marketing database (or CRM in this case) is as valuable to me as any campaign I can dream of, as the strength of my campaign is only as good as the quality of data used to design it.

Locating and addressing the root cause of data quality issues is paramount in conducting efficient business operations in any organizational department.  Here’s a quick framework of how I do it:

1) Define the problem. Without trying to solve it, I make a statement as to what the problem is. What is wrong with what and include the frequency and isolation of occurrence if possible.

2) Gather facts. Again, without trying to solve the problem, I collect as much data ABOUT the problem as I can, asking other team members for input and looking for patterns as I go.

More Read

Using Your Human Database to Select a Database

Flip that Data!

Not Only SQL, Not Only Big Data

Delivering Data Happiness

The Journey from Big Data to Big Promise

3) Compare and relate. Do any of the facts I’ve gathered relate exclusively to the problem I’ve described? If yes, list them.

4) Determine probable causes. Use deductive reasoning to weed out possibilities and identify your probable root causes, keeping in mind you’ll likely have more than one.

5) Test and check. Test each probable cause and check results to see which causes the problem to occur.

Root Cause Analysis in Practice

Here’s the theory in practice.  Imagine I need to build a contact information report for my sales team that is based on geographic territories. I set up the report and hit run, but the report only shows 10% of records with a state field.

1) Problem: Contact information report is missing state field on 90% of records.

2) Facts: There are no known performance issues with the database. The contract information report is built correctly (pulling the desired data fields with the desired filters). State fields were imported recently and this is the first time I’ve ran a report using State as a field.

3) Exclusive Facts: The contract information report is built correctly (pulling the desired data fields with the desired filters). State fields were imported recently and this is the first time I’ve ran a report using State as a field.

4) Probable cause: State fields were not imported correctly.

5) Test: Create a file to import with test data where state should be. Import the file and check results. Still not imported correctly? This is your likely root cause.

For more assistance with locating, and more importantly, addressing data quality issues, MIKE2.0 offers an open source solution to help address and correct them. Check it out when you have a moment.