Big Data Hype is an Opportunity for Data Management Pros

Big Data is a hot topic in the data management world.

Big Data is a hot topic in the data management world. Recently, I’ve seen press and vendors describing it with the words crucial, tremendous opportunity, overcoming vexing challenges, and enabling technology.  With all the hoopla, management is probably asking many of you about your Big Data strategy. It has risen to the corporate management level; your CxO is probably aware.

Most of the data management professionals I’ve met are fairly down-to-earth, pragmatic folks.  Data is being managed correctly or not. The business rule works, or it does not. Marketing spin is evil. In fact, the hype and noise around big data may be something to be filtered by many of you. You’re appropriately trying to look through the hype and get to the technology or business process that’s being enhanced by Big Data.

However, in addition to filtering through the big data hype to the IT impact, data management professionals should also embrace the hype.

Sure, we want to handle the high volume transactions that often come with big data, but we still have relational databases and unstructured data sources to deal with.  We still have business users using Excel for databases with who-knows-what in them.  We still have e-mail attachments from partners that need to be incorporated into our infrastructure.  We still have a wide range of data sources and targets that we have to deal with, including, but not limited to, big data. In my last blog post, I wrote about how big data is just one facet of total data management.

The opportunity is for data management pros to think about their big data management strategy holistically and solve some of their old and tired issues around data management. It’s pretty easy to draw a picture for management that Big Data needs to take a Total Data Management approach.  An approach that includes some of our worn-out and politically-charged data governance issues, including:

• Data Ownership – One barrier to big data management is accountability for the data.  By deciding you are going to plan for big data, you also need to make decisions about who owns the big data, and all your data sets for that matter.
• Spreadmarts – Keeping unmanaged data out of spreadsheets is increasingly more crucial in companies who must handle Big Data. So-called “spreadmarts,” which are important pieces of data stored in Excel spreadsheets, are easily replicated to team desktops. In this scenario, you lose control of versions as well as standards. However, big data can help make it easy for everyone to use corporate information, no matter what size.
• Unstructured Data – Although big data might tend be more analytical than operational, big data is most commonly unstructured data.  A total data management approach takes into account unstructured data in either case. Having technology and processes that handles unstructured data, big or small, is crucial to total data management.
• Corporate Strategy and Mergers – If your company is one that grows through acquisition, managing big data is about being able to handle, not only your own data, but the data of those companies you acquire.  Since you don’t know what systems those companies will have, a big data governance strategy and flexible tools are important to big data.

My point is, with big data, try to avoid the typical noise filtering exercises you normally take on the latest buzzword.  Instead, use the hype and buzz to your advantage to address a holistic view of data management in your organization.

Covering the world of data integration, data governance, and data quality from the perspective of an industry insider.