We use cookies, including third-party cookies from Google to serve personalized ads through AdSense, to operate this site and understand how it is used. By continuing to browse, you accept this use. See our Privacy Policy and Terms of Use for details, including how to opt out of personalized advertising.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    Flat editorial illustration: The article is about joining public web signals with internal business metrics so BI teams can inter
    Alternative Data in BI: Bringing Public Web Data Into Your Dashboards
    23 Min Read
    Flat editorial illustration: The article is about automating the collection, preparation, and delivery of marketing data across p
    From Data to Dashboards: 5 Marketing Reporting Automation Software Picks
    12 Min Read
    Flat editorial illustration: The article contrasts passive follower totals with a behavioral path from attention and trust to cli
    Creator Analytics Beyond Follower Count: Seven Metrics That Measure Audience Behavior
    11 Min Read
    Flat editorial illustration: The article connects AI and marketing analytics with managerial judgment grounded in consumer behavi
    How an Online MBA in Marketing Connects AI and Data Analytics
    11 Min Read
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers -- AI-generated illustration
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers
    7 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: Heal the Heartbreak of Data Sprawl with a Data Catalog
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Data Management > Best Practices > Heal the Heartbreak of Data Sprawl with a Data Catalog
Best PracticesBig DataBusiness IntelligenceData ManagementExclusiveIT

Heal the Heartbreak of Data Sprawl with a Data Catalog

AndrewAhn
AndrewAhn
5 Min Read
Heal the Heartbreak of Data Sprawl with a Data Catalog
Illustration generated with Qwen Image.
SHARE

Your security analytics team wants a copy of your production database so they can look for fraudulent accounts. Your accounts payable department wants an extract it can analyze to improve supply chain efficiency. Your sales manager wants all your customer records so he can merge them with his Salesforce.com data. And your database administrator is using both snapshots and two full backups just to be sure all the data is safe.

Data Sprawl Happens when Data is Needlessly Duplicated

What you’ve got is a typical data sprawl problem in the making. That’s what happens when organizations – for whatever reasons – create multiple copies of production data. There’s always a good reason for each copy to be created, but collectively they become a mess.

Data sprawl is becoming a real problem as business users increasingly want to analyze data themselves, within the context of big data. International Data Corp. has estimated that up to 60% of total storage capacity is now dedicated to accommodating copy data, and that the total cost of copy data storage will top $50 billion next year. Yet it estimates that fewer than 20% of organizations have copy management standards. Gartner analyst Dave Russell says many companies keep between 30 and 40 copies of business data.

Data Sprawl Leads to Organizations Falling Out-of-Sync

In addition to the obvious toll that data sprawl takes on infrastructure and performance, data integrity becomes a real problem. For example, a salesperson making an update to a customer record in the CRM system risks being out of sync with the same record in the customer database. A database administrator who restores the wrong backup may overwrite production data with old information.

More Read

Understanding the Basics of Google Analytics
Understanding the Basics of Google Analytics
Banks Merge Data Mining and CRM Tools to Boost Profitability
AI Boosts Business Productivity with Open Source Calendars
DIY Culture: Should Non-IT Employees Be Compensated for Building Apps?
What Cyber Criminals Can Do With Your Metadata

Numerous companies are developing costly technology-based solutions to the copy sprawl problem, but for many customer organizations, the simplest and most cost-effective approach is good data governance grounded in a data catalog.

An enterprise data catalog maintains a single directory of all the data the company owns. This can include not only production data, but also backups, extracts, and summaries. Production data can be “fingerprinted” with a unique signature so that out-of-date copies never inadvertently make their way into mission-critical applications. Similarly, copies and extracts can be tagged according to their intended use. A catalog can even improve data integrity by ensuring that data marked with certain meta tags is never overwritten.

Data Catalogs Plus Strong Data Governance Policies are the Solution

Use of a data catalog should be combined with good governance practices. For example, employees need to know what data is okay for analytical use and what shouldn’t be touched; which are copies or new relevant data.   Database administrators need clear parameters on how to restore backed up data sets. One way to make data governance both effective and enjoyable is to encourage business users to join in the process by tagging their own data through a crowdsourced data quality program.

Using a data catalog eases the infrastructure penalty of data sprawl by reducing the incidence of orphaned data. It can also reduce the burden on database administrators while actually increasing responsiveness to business user requests. For example, the sales manager who needs customer records can use a catalog to find a satisfactory database that already exists in another department and avoid joining a backlog of IT job tickets.

Businesses shouldn’t suffer because of too much internal demand for data. The solution isn’t to deny requests with an agility-killing gatekeeping process, but to better understanding what data you have so that it will be more useful.  The curation and governance that a proper catalog can provide — that’s cure for data sprawl and the path to a data driven company.

TAGGED:big databusiness intelligence
Share This Article
Facebook Pinterest LinkedIn
Share
ByAndrewAhn
Follow:
Andrew Ahn is Vice President of Product Management for Waterline Data. He is an Apache Atlas committer and was the lead at Hortonworks for Hadoop governance strategy. Prior work includes product and governance responsibilities at ICE/NYSE Euronext, spanning 12 countries and 23 market centers.

Follow us on Facebook

Latest News

Flat editorial illustration: The article is about joining public web signals with internal business metrics so BI teams can inter
Alternative Data in BI: Bringing Public Web Data Into Your Dashboards
Big Data Business Intelligence Exclusive Web Analytics
Flat editorial illustration: A SaaS-editorial scene showing product usage, support patterns, and renewal feedback informing a fle
Using Product Usage Analytics and Support Data to Build a B2B Product Roadmap
Business Intelligence Infographic
From Hiring To Payroll: Managing HR Data Across Business Systems -- AI-generated illustration
From Hiring To Payroll: Managing HR Data Across Business Systems
Big Data Business Intelligence Infographic
Flat editorial illustration: The article is about automating the collection, preparation, and delivery of marketing data across p
From Data to Dashboards: 5 Marketing Reporting Automation Software Picks
Analytics Business Intelligence Exclusive Marketing

Stay Connected

1.2KFollowersLike
33.7KFollowersFollow
222FollowersPin

You Might also Like

Datameer Provides End-user Focused BI Solutions for Big Data Analytics
AnalyticsBusiness IntelligenceCloud ComputingHadoop

Datameer Provides End-user Focused BI Solutions for Big Data Analytics

4 Min Read
5 Questions To Ask Before Embedding Business Intelligence Into Software
Business IntelligenceExclusiveSoftware

5 Questions To Ask Before Embedding Business Intelligence Into Software

7 Min Read
3 Ways Fintech Is Using Big Data to Reshape Lending
Big DataExclusiveFintechNews

3 Ways Fintech Is Using Big Data to Reshape Lending

6 Min Read
The Cloud Doesn't Eliminate The Need For Data Recovery Software
Cloud ComputingExclusiveITSoftware

The Cloud Doesn’t Eliminate The Need For Data Recovery Software

6 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

From Bolts to Bots: How AI Is Fortifying the Automotive Industry
From Bolts to Bots: How AI Is Fortifying the Automotive Industry
Artificial Intelligence
The Art of Conversation: Enhancing Chatbots with Advanced AI Prompts
The Art of Conversation: Enhancing Chatbots with Advanced AI Prompts
Chatbots

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-26 SmartData Collective. All Rights Reserved.
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?