4 Ways to Prevent Dirty Data From Spoiling Analytics

Change is good — except when change means you are constantly revising and checking databases, unsure of whether records are accurate, current, complete and useful.  Does the data allow you to do something?  If not, all you have are souvenirs from bygone days.

Change is good — except when change means you are constantly revising and checking databases, unsure of whether records are accurate, current, complete and useful.  Does the data allow you to do something?  If not, all you have are souvenirs from bygone days. Useful data can be contacts, marketing reports, sales figures, projections, analytics of past actions or predictive patterns for future decisions.  But, like lighting matches, some data records are single-use only and quickly obsolete. Finding details that can be recycled and reused is a greater challenge. 

Our recent post on ‘dirty data‘ got a lot of response and is another example of how analytics and business intelligence relies on content in proper context. So here are four actions you can take to improve the quality and usability of your data:

1. Decide who is in charge – Put a team together that has final authority for data hygiene or has authority to resolve conflicts over whose information is correct. Depending on your industry, regulatory controls and other issues, it may be necessary to have Sales, IT, Legal and other departments represented. When your team has an analytics and data use policy it may be necessary to consider client privacy issues, changes in access levels or even a ‘mediation’ system for deciding which database is used by which departments or operating groups.

2. Test Your Data and Programming – Savvy marketers check the bounce rates and reasons for non-delivery in testing email contacts. Look for ways to double-check your data and correlate it with other relevant information that can be useful. Set a schedule for data checking that fits your corporate need (daily, monthly and quarterly) and see how data flow and accuracy improves. Who might use that increased data and help fill in the blanks or look for the “Why” that explains the data?

3. Find An Outside Expert – Is there another company or service provider with industry averages or specific content that can help verify your information, provide context or comparison that makes your data more valuable. Linked In sends you a network update on people who have changed jobs or updated their profiles. Can your data do the same thing, reporting changes since the last report or comparing different geographic areas. Data Service Providers such as OneSource, ZoomInfo, SalesForce or Hoovers may have details that compliment your existing data records on people and companies.

4. Rinse, Lather Repeat – Your work is never done. So it may take some time to get your data to a higher level of cleanliness. Measure your progress and report your activity to get more participation. Consider ongoing programs to discard data after a specific amount of time, update a record that hasn’t been touched in, say, six months and consistently work to improve the accuracy of your data. Are your projections on-target or are they unable to provide a clear forecast?

Better data means more accurate analytics and more targeted business intelligence. Providing that valuable service — along with squeaky-clean data — could make you indispensable. And we could all use that kind of reaction.

David Wallace
Spotfire Blogging Team