Your security analytics team wants a copy of your production database so they can look for fraudulent accounts. Your accounts payable department wants an extract it can analyze to improve supply chain efficiency. Your sales manager wants all your customer records so he can merge them with his Salesforce.com data. And your database administrator is using both snapshots and two full backups just to be sure all the data is safe.
Data Sprawl Happens when Data is Needlessly Duplicated
What you’ve got is a typical data sprawl problem in the making. That’s what happens when organizations – for whatever reasons – create multiple copies of production data. There’s always a good reason for each copy to be created, but collectively they become a mess.
Data sprawl is becoming a real problem as business users increasingly want to analyze data themselves, within the context of big data. International Data Corp. has estimated that up to 60% of total storage capacity is now dedicated to accommodating copy data, and that the total cost of copy data storage will top $50 billion next year. Yet it estimates that fewer than 20% of organizations have copy management standards. Gartner analyst Dave Russell says many companies keep between 30 and 40 copies of business data.
Data Sprawl Leads to Organizations Falling Out-of-Sync
In addition to the obvious toll that data sprawl takes on infrastructure and performance, data integrity becomes a real problem. For example, a salesperson making an update to a customer record in the CRM system risks being out of sync with the same record in the customer database. A database administrator who restores the wrong backup may overwrite production data with old information.
Numerous companies are developing costly technology-based solutions to the copy sprawl problem, but for many customer organizations, the simplest and most cost-effective approach is good data governance grounded in a data catalog.
An enterprise data catalog maintains a single directory of all the data the company owns. This can include not only production data, but also backups, extracts, and summaries. Production data can be “fingerprinted” with a unique signature so that out-of-date copies never inadvertently make their way into mission-critical applications. Similarly, copies and extracts can be tagged according to their intended use. A catalog can even improve data integrity by ensuring that data marked with certain meta tags is never overwritten.
Data Catalogs Plus Strong Data Governance Policies are the Solution
Use of a data catalog should be combined with good governance practices. For example, employees need to know what data is okay for analytical use and what shouldn’t be touched; which are copies or new relevant data. Database administrators need clear parameters on how to restore backed up data sets. One way to make data governance both effective and enjoyable is to encourage business users to join in the process by tagging their own data through a crowdsourced data quality program.
Using a data catalog eases the infrastructure penalty of data sprawl by reducing the incidence of orphaned data. It can also reduce the burden on database administrators while actually increasing responsiveness to business user requests. For example, the sales manager who needs customer records can use a catalog to find a satisfactory database that already exists in another department and avoid joining a backlog of IT job tickets.
Businesses shouldn’t suffer because of too much internal demand for data. The solution isn’t to deny requests with an agility-killing gatekeeping process, but to better understanding what data you have so that it will be more useful. The curation and governance that a proper catalog can provide — that’s cure for data sprawl and the path to a data driven company.