By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData CollectiveSmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    data-driven white label SEO
    Does Data Mining Really Help with White Label SEO?
    7 Min Read
    marketing analytics for hardware vendors
    IT Hardware Startups Turn to Data Analytics for Market Research
    9 Min Read
    big data and digital signage
    The Power of Big Data and Analytics in Digital Signage
    5 Min Read
    data analytics investing
    Data Analytics Boosts ROI of Investment Trusts
    9 Min Read
    football data collection and analytics
    Unleashing Victory: How Data Collection Is Revolutionizing Football Performance Analysis!
    4 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: Data Error Inequality
Share
Notification Show More
Aa
SmartData CollectiveSmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Data Management > Best Practices > Data Error Inequality
AnalyticsBest Practices

Data Error Inequality

MIKE20
Last updated: 2011/03/16 at 9:21 PM
MIKE20
4 Min Read
SHARE

Contents
A Very Simple ModelThe bottom line from the table above is that not all data issues are created equal. Brass tacks: missing or erroneous information in a customer, vendor, or employee master record is not the same as an “information-only” field that drives nothing. (For more information on this, see the MIKE2.0 Master Data Management Offering.)Simon SaysFeedback

On his excellent data quality and management blog, my friend Henrik Liliendahl recently wrote an excellent post entitled “Good, fast, cheap – pick any two.” In his post, Henrik discusses the well-worn trade-off between among things well, quickly, and for very little money. To quote Henrik:

More Read

data security

NIST 800-171 Safeguards Help Non-Federal Networks Handling CUI

Does Data Mining Really Help with White Label SEO?
IT Hardware Startups Turn to Data Analytics for Market Research
The Power of Big Data and Analytics in Digital Signage
Data Analytics Boosts ROI of Investment Trusts

Some data, especially those we call master data, is used for multiple purposes within an organization. Therefore some kind of real world alignment is often used as a fast track to improving data quality where you don’t spend time analyzing how data may fit multiple purposes at the same time in your organization. Real world alignment also may fulfill future requirements regardless of the current purposes of use.

As usual, Henrik is absolutely right and many consultants have heard the axiom on which his post is based.

A Very Simple Model

In my day, I have seen people grossly overreact to data quality or conversion issues. Generally speaking, I have seen three types of errors. Note that this is a very simple model and cannot possibly account for every type of scenario and potentially pernicious downstream effect:

Type of ErrorExampleShould You Freak Out?
Master Record ErrorCustomer or EmployeeProbably
Important Characteristic Field ErrorEmployee AddressKind of
Information or “Nice to Have” Field ErrorCustomer backup contact numberNo

The bottom line from the table above is that not all data issues are created equal. Brass tacks: missing or erroneous information in a customer, vendor, or employee master record is not the same as an “information-only” field that drives nothing. (For more information on this, see the MIKE2.0 Master Data Management Offering.)

Of course, not everyone understands this. I can think of one woman (call her Dorothy here) with whom I worked on an enormous ERP project. To her, all errors were major issues. For example, I remember when the consulting team of which I was a part ran conversion programs attempting to load more than one million historical records into the new payroll system. Something like 12,000 records were flagged as potential issues.

Do the math. That’s nearly a 99 percent accuracy rate–and the data was much, much better coming out of the legacy system based upon a very sophisticated ETL tool created to minimize those errors. Further, the vast majority of those errors were “information only” soft edits from the vendor’s conversion program. That is, they weren’t really errors.

Of course, Dorothy chose to focus on (in her view) the enormity of the 12,000 number. She did not want to hear explanations. While this irritated me (given how how the team had been working to cleanse the client’s legacy data), I wasn’t all that surprised. Dorothy knew nothing about data management and this was her first experience managing a project anywhere near this scope.

Simon Says

Fight the urge to treat all errors and issues as equal. They are not. Take the time to understand the nuances of your data, your information management project’s constraints, and the links among different systems, tables, and applications. You’ll find that your team will respect you more if you invest a few minutes in separating major issues from non-issues.

Feedback

What say you?

MIKE20 March 16, 2011
Share This Article
Facebook Twitter Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

big data and IP laws
Big Data & AI In Collision Course With IP Laws – A Complete Guide
Big Data
ai in marketing
4 Ways AI Can Enhance Your Marketing Strategies
Marketing
sobm for ai-driven cybersecurity
Software Bill of Materials is Crucial for AI-Driven Cybersecurity
Security
IT budgeting for data-driven companies
IT Budgeting Practices for Data-Driven Companies
IT

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

data security
Data Management

NIST 800-171 Safeguards Help Non-Federal Networks Handling CUI

5 Min Read
data-driven white label SEO
Analytics

Does Data Mining Really Help with White Label SEO?

7 Min Read
marketing analytics for hardware vendors
Analytics

IT Hardware Startups Turn to Data Analytics for Market Research

9 Min Read
big data and digital signage
Analytics

The Power of Big Data and Analytics in Digital Signage

5 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

AI chatbots
AI Chatbots Can Help Retailers Convert Live Broadcast Viewers into Sales!
Chatbots
ai is improving the safety of cars
From Bolts to Bots: How AI Is Fortifying the Automotive Industry
Artificial Intelligence

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?