By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData Collective
  • Analytics
    AnalyticsShow More
    data analytics in sports industry
    Here’s How Data Analytics In Sports Is Changing The Game
    6 Min Read
    data analytics on nursing career
    Advances in Data Analytics Are Rapidly Transforming Nursing
    8 Min Read
    data analytics reveals the benefits of MBA
    Data Analytics Technology Proves Benefits of an MBA
    9 Min Read
    data-driven image seo
    Data Analytics Helps Marketers Substantially Boost Image SEO
    8 Min Read
    construction analytics
    5 Benefits of Analytics to Manage Commercial Construction
    5 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: Responding to a Follower’s Question: Why Keep Data Replication to a Minimum?
Share
Notification Show More
Latest News
data analytics in sports industry
Here’s How Data Analytics In Sports Is Changing The Game
Big Data
data analytics on nursing career
Advances in Data Analytics Are Rapidly Transforming Nursing
Analytics
data analytics reveals the benefits of MBA
Data Analytics Technology Proves Benefits of an MBA
Analytics
anti-spoofing tips
Anti-Spoofing is Crucial for Data-Driven Businesses
Security
ai in software development
3 AI-Based Strategies to Develop Software in Uncertain Times
Software
Aa
SmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Big Data > Data Mining > Responding to a Follower’s Question: Why Keep Data Replication to a Minimum?
Data MiningData QualityData Visualization

Responding to a Follower’s Question: Why Keep Data Replication to a Minimum?

Rob Armstrong
Last updated: 2011/05/11 at 12:59 PM
Rob Armstrong
4 Min Read
SHARE

I got an e-mail from one of my followers (I say this in the hopes there are many more!).  In my blog post “What will you tolerate?” I provided a sample of guiding principles.  One of them was the suggestion that data replication be kept to a minimum.  The reader wanted to get a bit more depth to that point.

I got an e-mail from one of my followers (I say this in the hopes there are many more!).  In my blog post “What will you tolerate?” I provided a sample of guiding principles.  One of them was the suggestion that data replication be kept to a minimum.  The reader wanted to get a bit more depth to that point.

Going with the theory that if one person in the audience has that question there may be several more with the same thought, I wanted to just clarify and expand on that point.  I will give the shout out to Jon and thank him for the question (as well as correcting some of my typos).

More Read

data visualization for small business

Data Visualization Boosts Business Scalability with Sales Mapping

Use this Strategic Approach to Maximize Your Data’s Value
Niche Data Tactics to Take Your Business to the Next Level
5 Best Practices for Extracting, Analyzing, and Visualizing Data
Can Data Mining Aid with Off-Page SEO Strategies?

Editor’s note: Rob Armstrong is an employee of Teradata. Teradata is a sponsor of The Smart Data Collective.

So why do I suggest that data replication be minimized.  There are several reasons beyond the very obvious one of disk storage and cost to maintain.

The main point about this guiding principle is that once the data has been cleansed, transformed, and integrated into the core data warehouse, the access should be against that data directly (or through views).  There is very little reason to then extract the data to another database or platform for analytics.  Many people will extract the data into data marts, excel, or other applications. 

Often times this is justified by claiming performance factors, IT barriers, or a variety of other issues.

Whatever the reason, this duplication of data is a problem and should be avoided when possible.  When data is replicated out very rarely do the data rules, data quality, and auditing trails accompany the extract.  This leads to users taking data, possibly transforming it in their on spreadsheets and then sharing that extract with others.  Now the data in the data warehouse no longer matches any reports or analytics from the extracts.  This lead to confusion and finger pointing about where answers are coming from and who’s answers are correct.  Added to this is a problem when a user decided they want to “drill down” from manipulated data but the underlying data in the warehouse no longer matches the reports.

Now this is not to say there is never a time that replicating data is justified.  Clearly, you will need to replicate data for disaster recovery systems.  You may also want to replicate data into a test environment so new applications can be developed and tested against “real data”.  These cases are reasonable as the data is audited for consistency and do not become the source of new analytics.

You may also have to overcome a real technical issue such as an business critical (with proven value) application that requires data to be co-located with the process.  In this case care should be taken to really document what the technical issue is and how it needs to be resolved.  Finally, there needs to be the understanding that the application will be pointed back to the core warehouse once the issue is resolved.  This of course leads us to the whole role of governance but that is another blog.

That help?

Rob Armstrong May 11, 2011
Share this Article
Facebook Twitter Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

data analytics in sports industry
Here’s How Data Analytics In Sports Is Changing The Game
Big Data
data analytics on nursing career
Advances in Data Analytics Are Rapidly Transforming Nursing
Analytics
data analytics reveals the benefits of MBA
Data Analytics Technology Proves Benefits of an MBA
Analytics
anti-spoofing tips
Anti-Spoofing is Crucial for Data-Driven Businesses
Security

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

data visualization for small business
Data Visualization

Data Visualization Boosts Business Scalability with Sales Mapping

7 Min Read
analyzing big data for its quality and value
Big Data

Use this Strategic Approach to Maximize Your Data’s Value

6 Min Read
niche data tactics for business success
Big Data

Niche Data Tactics to Take Your Business to the Next Level

6 Min Read
big data visualization
Data Visualization

5 Best Practices for Extracting, Analyzing, and Visualizing Data

6 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

AI and chatbots
Chatbots and SEO: How Can Chatbots Improve Your SEO Ranking?
Artificial Intelligence Chatbots Exclusive
giveaway chatbots
How To Get An Award Winning Giveaway Bot
Big Data Chatbots Exclusive

Quick Link

  • About
  • Contact
  • Privacy
Follow US

© 2008-23 SmartData Collective. All Rights Reserved.

Removed from reading list

Undo
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?