Cookies help us display personalized product recommendations and ensure you have great shopping experience.

By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    big data analytics in transporation
    Turning Data Into Decisions: How Analytics Improves Transportation Strategy
    3 Min Read
    sales and data analytics
    How Data Analytics Improves Lead Management and Sales Results
    9 Min Read
    data analytics and truck accident claims
    How Data Analytics Reduces Truck Accidents and Speeds Up Claims
    7 Min Read
    predictive analytics for interior designers
    Interior Designers Boost Profits with Predictive Analytics
    8 Min Read
    image fx (67)
    Improving LinkedIn Ad Strategies with Data Analytics
    9 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: Data Error Inequality
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Data Management > Best Practices > Data Error Inequality
AnalyticsBest Practices

Data Error Inequality

MIKE20
MIKE20
4 Min Read
SHARE

Contents
  • A Very Simple Model
  • The bottom line from the table above is that not all data issues are created equal. Brass tacks: missing or erroneous information in a customer, vendor, or employee master record is not the same as an “information-only” field that drives nothing. (For more information on this, see the MIKE2.0 Master Data Management Offering.)
  • Simon Says
  • Feedback

On his excellent data quality and management blog, my friend Henrik Liliendahl recently wrote an excellent post entitled “Good, fast, cheap – pick any two.” In his post, Henrik discusses the well-worn trade-off between among things well, quickly, and for very little money. To quote Henrik:

More Read

Probability and Karl Rove
Are You Sweeping Big Data Privacy Under the Carpet? 5 Things to Do Instead
REvolution Computing welcomes Danese Cooper
Business People Are Dumb On Average(s)
The Hegemony of Large Numbers – Ignoring Common Sense

Some data, especially those we call master data, is used for multiple purposes within an organization. Therefore some kind of real world alignment is often used as a fast track to improving data quality where you don’t spend time analyzing how data may fit multiple purposes at the same time in your organization. Real world alignment also may fulfill future requirements regardless of the current purposes of use.

As usual, Henrik is absolutely right and many consultants have heard the axiom on which his post is based.

A Very Simple Model

In my day, I have seen people grossly overreact to data quality or conversion issues. Generally speaking, I have seen three types of errors. Note that this is a very simple model and cannot possibly account for every type of scenario and potentially pernicious downstream effect:

Type of ErrorExampleShould You Freak Out?
Master Record ErrorCustomer or EmployeeProbably
Important Characteristic Field ErrorEmployee AddressKind of
Information or “Nice to Have” Field ErrorCustomer backup contact numberNo

The bottom line from the table above is that not all data issues are created equal. Brass tacks: missing or erroneous information in a customer, vendor, or employee master record is not the same as an “information-only” field that drives nothing. (For more information on this, see the MIKE2.0 Master Data Management Offering.)

Of course, not everyone understands this. I can think of one woman (call her Dorothy here) with whom I worked on an enormous ERP project. To her, all errors were major issues. For example, I remember when the consulting team of which I was a part ran conversion programs attempting to load more than one million historical records into the new payroll system. Something like 12,000 records were flagged as potential issues.

Do the math. That’s nearly a 99 percent accuracy rate–and the data was much, much better coming out of the legacy system based upon a very sophisticated ETL tool created to minimize those errors. Further, the vast majority of those errors were “information only” soft edits from the vendor’s conversion program. That is, they weren’t really errors.

Of course, Dorothy chose to focus on (in her view) the enormity of the 12,000 number. She did not want to hear explanations. While this irritated me (given how how the team had been working to cleanse the client’s legacy data), I wasn’t all that surprised. Dorothy knew nothing about data management and this was her first experience managing a project anywhere near this scope.

Simon Says

Fight the urge to treat all errors and issues as equal. They are not. Take the time to understand the nuances of your data, your information management project’s constraints, and the links among different systems, tables, and applications. You’ll find that your team will respect you more if you invest a few minutes in separating major issues from non-issues.

Feedback

What say you?

Share This Article
Facebook Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

ai for building crypto banks
Building Your Own Crypto Bank with AI
Blockchain Exclusive
julia taubitz vn5s g5spky unsplash
Benefits of AI in Nursing Education Amid Medicaid Cuts
Artificial Intelligence Exclusive News
AI role in medical industry
The Role Of AI In Transforming Medical Manufacturing
Artificial Intelligence Exclusive
b2b sales
Unseen Barriers: Identifying Bottlenecks In B2B Sales
Business Rules Exclusive Infographic

Stay Connected

1.2kFollowersLike
33.7kFollowersFollow
222FollowersPin

You Might also Like

Paul Kedrosky, “the man who counts ladders”

3 Min Read
Image
AnalyticsBig DataIT

IoT hits pay dirt where needs and capabilities align

7 Min Read
big data in preventative care
AnalyticsBig DataExclusive

Big Data For Preventative Care In The Healthcare Field

4 Min Read

Target, Pregnancy, and Predictive Analytics – Part I

5 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

AI and chatbots
Chatbots and SEO: How Can Chatbots Improve Your SEO Ranking?
Artificial Intelligence Chatbots Exclusive
data-driven web design
5 Great Tips for Using Data Analytics for Website UX
Big Data

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-25 SmartData Collective. All Rights Reserved.
Go to mobile version
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?