By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData CollectiveSmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    customer experience analytics
    Using Data Analysis to Improve and Verify the Customer Experience and Bad Reviews
    6 Min Read
    data analytics and CRO
    Data Analytics is Crucial for Website CRO
    9 Min Read
    analytics in digital marketing
    The Importance of Analytics in Digital Marketing
    8 Min Read
    benefits of investing in employee data
    6 Ways to Use Data to Improve Employee Productivity
    8 Min Read
    Jira and zendesk usage
    Jira Service Management vs Zendesk: What Are the Differences?
    6 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: Schrödinger’s Data Quality
Share
Notification Show More
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Uncategorized > Schrödinger’s Data Quality
Uncategorized

Schrödinger’s Data Quality

JimHarris
Last updated: 2009/05/21 at 1:51 AM
JimHarris
6 Min Read
SHARE

In 1935, Austrian physicist Erwin Schrödinger described a now famous thought experiment where:

“A cat, a flask containing poison, a tiny bit of radioactive substance and a Geiger counter are placed into a sealed box for one hour. If the Geiger counter doesn’t detect radiation, then nothing happens and the cat lives. However, if radiation is detected, then the flask is shattered, releasing the poison which kills the cat. According to the Copenhagen interpretation of quantum mechanics, until the box is opened, the cat is simultaneously alive and dead. Yet, once you open the box, the cat will either be alive or dead, not a mixture of alive and dead.” 

This was only a thought experiment. Therefore, no actual cat was harmed. 

This paradox of quantum physics, known as Schrödinger’s Cat, poses the question:

“When does a quantum system stop existing as a mixture of states and become one or the other?”

Unfortunately, data quality projects are not thought experiments. They are complex, time consuming and expensive enterprise initiatives. Typically, a data quality tool is purchased, expert consultants are hired to supplement staffing, production data is copied to a developme…

In 1935, Austrian physicist Erwin Schrödinger described a now famous thought experiment where:

“A cat, a flask containing poison, a tiny bit of radioactive substance and a Geiger counter are placed into a sealed box for one hour. If the Geiger counter doesn’t detect radiation, then nothing happens and the cat lives. However, if radiation is detected, then the flask is shattered, releasing the poison which kills the cat. According to the Copenhagen interpretation of quantum mechanics, until the box is opened, the cat is simultaneously alive and dead. Yet, once you open the box, the cat will either be alive or dead, not a mixture of alive and dead.” 

This was only a thought experiment. Therefore, no actual cat was harmed. 

This paradox of quantum physics, known as Schrödinger’s Cat, poses the question:

“When does a quantum system stop existing as a mixture of states and become one or the other?”

Unfortunately, data quality projects are not thought experiments. They are complex, time consuming and expensive enterprise initiatives. Typically, a data quality tool is purchased, expert consultants are hired to supplement staffing, production data is copied to a development server and the project begins. Until it is completed and the new system goes live, the project is a potential success or failure. Yet, once the new system starts being used, the project will become either a success or failure.

This paradox, which I refer to as Schrödinger’s Data Quality, poses the question:

“When does a data quality project stop existing as potential success or failure and become one or the other?”

Data quality projects should begin with the parallel and complementary efforts of drafting the business requirements while also performing a data quality assessment, which can help you:

  • Verify data matches the metadata that describes it
  • Identify potential missing, invalid and default values
  • Prepare meaningful questions for subject matter experts
  • Understand how data is being used
  • Prioritize critical data errors
  • Evaluate potential ROI of data quality improvements
  • Define data quality standards
  • Reveal undocumented business rules
  • Review and refine the business requirements
  • Provide realistic estimates for development, testing and implementation

Therefore, the data quality assessment assists with aligning perception with reality and gets the project off to a good start by providing a clear direction and a working definition of success.

However, a common mistake is to view the data quality assessment as a one-time event that ends when development begins. 

Projects should perform iterative data quality assessments throughout the entire development lifecycle, which can help you:

  • Gain a data-centric view of the project’s overall progress
  • Build data quality monitoring functionality into the new system
  • Promote data-driven development
  • Enable more effective unit testing
  • Perform impact analysis on requested enhancements (i.e. scope creep)
  • Record regression cases for testing modifications
  • Identify data exceptions that require suspension for manual review and correction
  • Facilitate early feedback from the user community
  • Correct problems that could undermine user acceptance
  • Increase user confidence that the new system will meet their needs

If you wait until the end of the project to learn if you have succeeded or failed, then you treat data quality like a game of chance.

And to paraphrase Albert Einstein:

“Do not play dice with data quality.”

Link to original post

TAGGED: data quality
JimHarris May 21, 2009 May 21, 2009
Share This Article
Facebook Twitter Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

ai can help with nurse burnout
Breakthroughs in AI Are Helping to Prevent Nurse Burnout
Artificial Intelligence Exclusive
AI in marketing
AI Can’t Replace Creativity When Crafting Digital Content
Artificial Intelligence
ai in furniture design
Top 5 AI-Driven Furniture Engineering Design Applications
Artificial Intelligence
data protection regulation
Benefits of Data Management Regulations for Consumers & Businesses
Data Management

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

big data and agile
Big DataExclusive

Startups Use Data and Agile for Portfolio Management

5 Min Read
analyzing big data for its quality and value
Big Data

Use this Strategic Approach to Maximize Your Data’s Value

6 Min Read
data lineage tool
Big Data

7 Data Lineage Tool Tips For Preventing Human Error in Data Processing

6 Min Read
data quality and role of analytics
Data Quality

Preserving Data Quality is Critical for Leveraging Analytics with Amazon PPC

8 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

ai in ecommerce
Artificial Intelligence for eCommerce: A Closer Look
Artificial Intelligence
AI and chatbots
Chatbots and SEO: How Can Chatbots Improve Your SEO Ranking?
Artificial Intelligence Chatbots Exclusive

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?