We use cookies, including third-party cookies from Google to serve personalized ads through AdSense, to operate this site and understand how it is used. By continuing to browse, you accept this use. See our Privacy Policy and Terms of Use for details, including how to opt out of personalized advertising.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    chatgpt image jul 21, 2026, 04 34 30 pm
    4 Core Benefits of Predictive Maintenance after Vibration Analysis
    10 Min Read
    How Does Data Mining Boost Customer Satisfaction in Logistics? Harnessing Analytics for Results -- AI-generated illustration
    How Does Data Mining Boost Customer Satisfaction in Logistics? Harnessing Analytics for Results
    11 Min Read
    chatgpt image jul 13, 2026, 04 23 45 pm
    How Data Analytics Helps Companies Improve User Engagement
    19 Min Read
    chatgpt image jul 13, 2026, 03 59 46 pm
    How Data Analytics Improves Multi-Location Search Strategies
    10 Min Read
    cybersecurity efforts
    How Behavioral Analytics and AI Are Redefining Cybersecurity for Boca Raton Businesses
    14 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: Hyperactive Data Quality (Second Edition)
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Uncategorized > Hyperactive Data Quality (Second Edition)
Uncategorized

Hyperactive Data Quality (Second Edition)

JimHarris
JimHarris
10 Min Read
Hyperactive Data Quality (Second Edition)
Illustration generated with FLUX.2 [klein 4B] via Cloudflare Workers AI.
SHARE

In the first edition of Hyperactive Data Quality, I discussed reactive and proactive approaches using the data quality lake analogy from Thomas Redman’s excellent book Data Driven: Profiting from Your Most Important Business Asset:

Contents
  • Reactive Data Quality
  • Proactive Data Quality
  • Proactive Data Quality > Reactive Data Quality
  • An Arrow Thickly Smeared With Poison
  • “Flight to Data Quality”
  • Hyperactive Data Quality
  • Related Posts

“…a lake represents a database and the water therein the data. The stream, which adds new water, is akin to a business process that creates new data and adds them to the database. The lake… is polluted, just as the data are dirty. Two factories pollute the lake. Likewise, flaws in the business process are creating errors…

One way to address the dirty lake water is to clean it up… by running the water through filters, passing it through specially designed settling tanks, and using chemicals to kill bacteria and adjust pH.

The alternative is to reduce the pollutant at the point source – the factories.

The contrast between the two approaches is stark. In the first, the focus is on the lake; in the second, it is on the stream. So too with data. Finding and fixing errors focuses on the database and data that have already been created. Preventing errors focuses on the business processes and future data.”

Reactive Data Quality

Reactive Data Quality (i.e. “cleaning the lake” in Redman’s analogy) focuses entirely on finding and fixing the problems with existing data after it has been extracted from its sources. 

An obsessive-compulsive quest to find and fix every data quality problem is a laudable but ultimately unachievable pursuit (even for expert “lake cleaners”). Data quality problems can be very insidious and even the best “lake cleaning” process will still produce exceptions. Your process should be designed to identify and report exceptions when they occur. In fact, as a best practice, you should also include the ability to suspend incoming data that contain exceptions for manual review and correction.

Proactive Data Quality

Proactive Data Quality focuses on preventing errors at the sources where data is entered or received, and before it is extracted for use by downstream applications (i.e. “enters the lake” in Redman’s analogy). 

More Read

A Look Into How Data Centers Actually Work [INFOGRAPHIC]
A Look Into How Data Centers Actually Work [INFOGRAPHIC]
Data Mining in Stock Market
2 Critical Obstacles Facing Retailers for Data Driven Marketing
IBM Will Take Varicent for Sales Performance Management
See the Future with Your CRM

Redman describes the benefits of proactive data quality with what he calls the Rule of Ten:

“It costs ten times as much to complete a unit of work when the input data are defective (i.e., late, incorrect, missing, etc.) as it does when the input data are perfect.”

Proactive data quality advocates reevaluating business processes that create data, implementing improved controls on data entry screens and web forms, enforcing the data quality clause (you have one, right?) of your service level agreements with external data providers, and understanding the information needs of your consumers before delivering enterprise data for their use.

Proactive Data Quality > Reactive Data Quality

Proactive data quality is clearly the superior approach.  Although it is impossible to truly prevent every problem before it happens, the more control that can be enforced where data originates, the better the overall quality will be for enterprise information. 

Reactive data quality essentially treats the symptoms without curing the disease. As Redman explains: “…the problem with being a good lake cleaner is that life never gets better… it gets worse as more data…conspire to mean there is more work every day.”

So why do the vast majority of data quality initiatives use a reactive approach?

An Arrow Thickly Smeared With Poison

In Buddhism, there is a famous parable:

A man was shot with an arrow thickly smeared with poison.  His friends wanted to get a doctor to heal him, but the man objected by saying:

“I will neither allow this arrow to be pulled out nor accept any medical treatment until I know the name of the man who wounded me, whether he was a nobleman or a soldier or a merchant or a farmer or a lowly peasant, whether he was tall or short or of average height, whether he used a long bow or a crossbow, and whether the arrow that wounded me was hoof-tipped or curved or barbed.” 

While his friends went off in a frantic search for these answers, the man slowly, and painfully, dies.

“Flight to Data Quality”

In economics, the term “flight to quality” describes the aftermath of a financial crisis (e.g., a stock market crash) when people become highly risk-averse and move their money into safer, more reliable investments.

A similar “flight to data quality” can occur in the aftermath of an event when poor data quality negatively impacted decision-critical enterprise information. Some examples include a customer service nightmare, a regulatory compliance failure, or a financial reporting scandal. 

Driven by a business triage for critical data problems, reactive data cleansing is purposefully chosen over proactive defect prevention. The priority is finding and fixing the near-term problems rather than worrying about the long-term consequences of not identifying the root cause and implementing process improvements that would prevent it from happening again.

The enterprise has been shot with an arrow thickly smeared with poison – poor data quality. Now is not the time to point out that the enterprise has actually shot itself by failing to have proactive measures in place. 

Reactive data quality only treats the symptoms. However, during triage, the priority is to stabilize the patient. A cure for the underlying condition is worthless if the patient dies before it can be administered.

Hyperactive Data Quality

Proactive data quality is the best practice. Root cause analysis, business process improvement, and defect prevention will always be more effective than the endlessly vicious cycle of reactive data cleansing. 

A data governance framework is necessary for proactive data quality to be successful. Patience and understanding are also necessary. Proactive data quality requires a strategic organizational transformation that will not happen easily or quickly. 

Even when not facing an immediate crisis, the reality is that reactive data quality will occasionally be a necessary evil that is used to correct today’s problems while proactive data quality is busy trying to prevent tomorrow’s problems.

Just like any complex problem, data quality has no fast and easy solution. Fundamentally, a hybrid discipline is required that combines proactive and reactive aspects into an approach that I refer to as Hyperactive Data Quality, which will make the responsibility for managing data quality a daily activity for everyone in your organization.

Please share your thoughts and experiences.

Related Posts

Hyperactive Data Quality (First Edition)

The General Theory of Data Quality

Link to original post

TAGGED:data quality
Share This Article
Facebook Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

8 MCP Tools for Market and Consumer Intelligence Workflows -- AI-generated illustration
8 MCP Tools for Market and Consumer Intelligence Workflows
Artificial Intelligence Exclusive
How Local Service Businesses Can Map Which Neighborhoods Generate the Most Revenue -- AI-generated illustration
How Local Service Businesses Can Map Which Neighborhoods Generate the Most Revenue
Business Intelligence
Best VMware Alternatives in Thailand for Private Cloud and HCI Deployments -- AI-generated illustration
Best VMware Alternatives in Thailand for Private Cloud and HCI Deployments
Cloud Computing Exclusive IT
Using Safety Metrics and Incident Data to Reduce Construction Risk and Insurance Costs -- AI-generated illustration
Using Safety Metrics and Incident Data to Reduce Construction Risk and Insurance Costs
Big Data Exclusive

Stay Connected

1.2KFollowersLike
33.7KFollowersFollow
222FollowersPin

You Might also Like

What To Know About The Impact of Data Quality and Quantity In AI
Artificial IntelligenceBig DataData QualityExclusiveMachine Learning

What To Know About The Impact of Data Quality and Quantity In AI

8 Min Read
On A Smarter Planet … Some Organizations Will Be Smarter-er Than Others
Uncategorized

On A Smarter Planet … Some Organizations Will Be Smarter-er Than Others

4 Min Read
Data Quality and the Cupertino Effect
Data Quality

Data Quality and the Cupertino Effect

7 Min Read
The Retail Data Nightmare: Coming to a Store Near You!
Business Intelligence

The Retail Data Nightmare: Coming to a Store Near You!

5 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

Chatbots and SEO: How Can Chatbots Improve Your SEO Ranking?
Chatbots and SEO: How Can Chatbots Improve Your SEO Ranking?
Artificial Intelligence Chatbots Exclusive
5 Great Tips for Using Data Analytics for Website UX
5 Great Tips for Using Data Analytics for Website UX
Big Data

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-26 SmartData Collective. All Rights Reserved.
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?