By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData Collective
  • Analytics
    AnalyticsShow More
    predictive analytics in dropshipping
    Predictive Analytics Helps New Dropshipping Businesses Thrive
    12 Min Read
    data-driven approach in healthcare
    The Importance of Data-Driven Approaches to Improving Healthcare in Rural Areas
    6 Min Read
    analytics for tax compliance
    Analytics Changes the Calculus of Business Tax Compliance
    8 Min Read
    big data analytics in gaming
    The Role of Big Data Analytics in Gaming
    10 Min Read
    analyst,women,looking,at,kpi,data,on,computer,screen
    Promising Benefits of Predictive Analytics in Asset Management
    11 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: Big Data – Quantity vs Quality: Don’t confuse lots of data with good data
Share
Notification Show More
Latest News
ai software development
Key Strategies to Develop AI Software Cost-Effectively
Artificial Intelligence
ai in omnichannel marketing
AI is Driving Huge Changes in Omnichannel Marketing
Artificial Intelligence
ai for small business tax planning
Maximize Tax Deductions as a Business Owner with AI
Artificial Intelligence
ai in marketing with 3D rendering
Marketers Use AI to Take Advantage of 3D Rendering
Artificial Intelligence
How Big Data Is Transforming the Maritime Industry
How Big Data Is Transforming the Maritime Industry
Big Data
Aa
SmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Big Data > Data Quality > Big Data – Quantity vs Quality: Don’t confuse lots of data with good data
AnalyticsData Quality

Big Data – Quantity vs Quality: Don’t confuse lots of data with good data

Rob Armstrong
Last updated: 2012/01/17 at 3:46 PM
Rob Armstrong
6 Min Read
SHARE

In the paper today (San Diego Union Tribune, Sunday 1/15) there was a good article on the need for data miners given the explosion of data that is currently occurring.  One point of note was that one of the biggest challenges is making data usable.  “If you do have the data you need, it is often very dirty”

In the paper today (San Diego Union Tribune, Sunday 1/15) there was a good article on the need for data miners given the explosion of data that is currently occurring.  One point of note was that one of the biggest challenges is making data usable.  “If you do have the data you need, it is often very dirty”

With that article, and all the talk surrounding the “BIG DATA” world, I recall one of the fundamental rules in the computing world – Garbage In, Garbage Out.  Anyone who has ever worked with data bases, application programming, or analytics can tell you that while having lots of data is good, having good data is lots better. 

More Read

How Big Data Is Transforming the Maritime Industry

How Big Data Is Transforming the Maritime Industry

Utilizing Data to Discover Shortcomings Within Your Business Model
Small Businesses Use Big Data to Offset Risk During Economic Uncertainty
The Importance of Data-Driven Approaches to Improving Healthcare in Rural Areas
How Big Data Is Transforming the Renewable Energy Sector

The problem is rumblings that some people think that just getting into the big data game will solve their woes, and that is not the case.  Many companies do not even have a handle on the “small data” (if I can coin a term here).  That is data that has structures that are relatively well known, the data is easily transformed and integrated into the corporate whole and has defined analytical purposes.  Surprising as it is, many companies are still struggling with the move from reports to analytics on this relatively small data.  Adding BIG DATA (not just volume but more so context and structure) into that mix can cause more confusion than clarity to the user community.

To just clarify my point, I certainly see the value and potential of incorporating big data into a complete picture of data.  I have long been a proponent for the inclusion of interaction data to a companies set of transaction data.  One can get a completely different picture of a customer by adding all sorts of data.  The point is that companies must understand that this creates a whole new set of data quality metrics and benchmarks as to what is clean enough of analytics and what is simply distractions and noise.

So how does one enter the big data pool without getting mired in the mud?  To start with I will provide a few of the steps I would take and then your questions and comments can determine if more needs to be posted.

First, ensure you have a good user community that is well steeped in the analytic process.  Are your users “looking at things” or “looking for things”.  If they are simply getting packaged reports, then adding big data will be more of a challenge.  In this situation, I would recommend starting with a small set of real data analysts (or now data scientist) that iteratively comb through sample sets of big data (social media, customer forums, web logs, etc) and see where are the missing opportunities or relationships which are currently unknown.  This insight can then be gradually cleansed and inserted into the user community by amending current reports and analytics.

Second, understand how the elements in your “big data” can correlate and join with the data in your enterprise.  By having the linkage between good solid corporate data and the contents of social media or customer reviews you can increase the value, and validity, of such data.  For example, getting feedback from a customer on products or experiences can be very useful.  It is even more useful if you can relate that information to the customers potential worth or past experiences such as purchases and returns.  You may not be able to stop the customer from posting comments but you can certainly get some context as to why the comments are being made.  All of this will increase aspects of data quality and allow you to address situations more completely.

Third, understand that much of the data that is coming in floods is not under your control.  The data is created in a multitude of environments, by an untold number of individuals, with little regard to rules or structure.  Trying to take that data and enforce quality may be more effort than it is worth.  By taking the data at face value and not attaching the importance that comes with “clean and audited” data you set the expectations in the right place, and that is much of the battle.  Where possible you can add filters and code consistency routines to improve the data quality. You can also use emerging tools to understand the semantics and emotion of free text and that will improve the quality and usage of that data.

Let’s start there and see where this goes.  I think there are going to be a lot of words written this year about big data.  Taking these simple steps upfront and preparing your company for the new challenges, and benefits, of this emerging arena will pay off for quite some time.

TAGGED: big data
Rob Armstrong January 17, 2012
Share this Article
Facebook Twitter Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

ai software development
Key Strategies to Develop AI Software Cost-Effectively
Artificial Intelligence
ai in omnichannel marketing
AI is Driving Huge Changes in Omnichannel Marketing
Artificial Intelligence
ai for small business tax planning
Maximize Tax Deductions as a Business Owner with AI
Artificial Intelligence
ai in marketing with 3D rendering
Marketers Use AI to Take Advantage of 3D Rendering
Artificial Intelligence

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

How Big Data Is Transforming the Maritime Industry
Big Data

How Big Data Is Transforming the Maritime Industry

8 Min Read
utlizing big data for business model
Big Data

Utilizing Data to Discover Shortcomings Within Your Business Model

6 Min Read
big data use in small businesses
Big Data

Small Businesses Use Big Data to Offset Risk During Economic Uncertainty

7 Min Read
data-driven approach in healthcare
Analytics

The Importance of Data-Driven Approaches to Improving Healthcare in Rural Areas

6 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

giveaway chatbots
How To Get An Award Winning Giveaway Bot
Big Data Chatbots Exclusive
ai is improving the safety of cars
From Bolts to Bots: How AI Is Fortifying the Automotive Industry
Artificial Intelligence

Quick Link

  • About
  • Contact
  • Privacy
Follow US

© 2008-23 SmartData Collective. All Rights Reserved.

Removed from reading list

Undo
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?