We use cookies, including third-party cookies from Google to serve personalized ads through AdSense, to operate this site and understand how it is used. By continuing to browse, you accept this use. See our Privacy Policy and Terms of Use for details, including how to opt out of personalized advertising.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    Flat editorial illustration: The article is about joining public web signals with internal business metrics so BI teams can inter
    Alternative Data in BI: Bringing Public Web Data Into Your Dashboards
    23 Min Read
    Flat editorial illustration: The article is about automating the collection, preparation, and delivery of marketing data across p
    From Data to Dashboards: 5 Marketing Reporting Automation Software Picks
    12 Min Read
    Flat editorial illustration: The article contrasts passive follower totals with a behavioral path from attention and trust to cli
    Creator Analytics Beyond Follower Count: Seven Metrics That Measure Audience Behavior
    11 Min Read
    Flat editorial illustration: The article connects AI and marketing analytics with managerial judgment grounded in consumer behavi
    How an Online MBA in Marketing Connects AI and Data Analytics
    11 Min Read
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers -- AI-generated illustration
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers
    7 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: It’s Time for a New Definition of Big Data
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Data Management > Culture/Leadership > It’s Time for a New Definition of Big Data
AnalyticsCollaborative DataCommentaryCulture/Leadership

It’s Time for a New Definition of Big Data

MIKE20
MIKE20
6 Min Read
It’s Time for a New Definition of Big Data
Photo by PublicDomainPictures on Pixabay (https://pixabay.com/photos/high-definition-multimedia-1827/)
SHARE

Two words seemingly on every technologist’s lips are “big data”.  The Wikipedia definition for big data is: “In information technology, big data consists of datasets that grow so large that they become awkward to work with using on-hand database management tools”.  This approach to describing the term constrains the discussion of big data to scale and fails to realise the key difference between regular data and big data.  The blog posts and books which cover the topic seem to converge on the same approach to defining big data and describe the challenges with extracting value from this resource in terms of its size.

Contents
  • Big data that is very small
  • Large datasets that aren’t big
  • Defining big data

Big data can really be very small and not all large datasets are big!  It’s time to find a new definition for big data.

Big data that is very small

Modern machines such as cars, trains, power stations and planes all have increasing numbers of sensors constantly collecting masses of data.  It is common to talk of having thousands or even hundreds of thousands of sensors all collecting information about the performance and activities of a machine.

Imagine a plane on a regular one hour flight with a hundred thousand sensors covering everything from the speed of air over every part of the airframe through to the amount of carbon dioxide in each section of the cabin.  Each sensor is effectively an independent device with its own physical characteristics.  The real interest is usually in combinations of sensor readings (such as carbon dioxide combined with cabin temperature and the speed of air combined with air pressure).  With so many sensors the combinations are incredibly complex and vary with the error tolerance and characteristics of individual devices.

More Read

How Does Big Data Change Social Media Marketing Strategies?
How Does Big Data Change Social Media Marketing Strategies?
How Should Companies Approach Big Data Discrimination?
The Role of Advanced Analytics in CRM
3 Crucial Ways Smart Data Eliminates Home Security Threats
Does It Take a Scientist to Find Gold in Big Data?

The data streaming from a hundred thousand sensors on an aircraft is big data.  However the size of the dataset is not as large as might be expected.  Even a hundred thousand sensors, each producing an eight byte reading every second would produce less than 3GB of data in an hour of flying (100,000 sensors x 60 minutes x 60 seconds x 8 bytes).  This amount of data would fit comfortably on a modest memory stick!

Large datasets that aren’t big

We are increasingly seeing systems that generate very large quantities of very simple data.  For instance, media streaming is generating very large volumes with increasing amounts of structured metadata.  Similarly, telecommunications companies have to track vast volumes of calls and internet connections.

Even if these two activities are combined, and petabytes of data is produced, the content is extremely structured.  As search engines, such as Google, and relational databases have shown, datasets can be parsed extremely quickly if the content is well structured.  Even though this data is large, it isn’t “big” in the same way as the data coming from the machine sensors in the earlier example.

Defining big data

If size isn’t what matters then what makes big data big?  The answer is in the number of independent data sources, each with the potential to interact.  Big data doesn’t lend itself well to being tamed by standard data management techniques simply because of its inconsistent and unpredictable combinations.

Another attribute of big data is its tendency to be hard to delete making privacy a common concern.  Imagine trying to purge all of the data associated with an individual car driver from toll road data.  The sensors counting the number of cars would no longer balance with the individual billing records which, in turn, wouldn’t match payments received by the company.

Perhaps a good definition of big data is to describe “big” in terms of the number of useful permutations of sources making useful querying difficult (like the sensors in an aircraft) and complex interrelationships making purging difficult (as in the toll road example).

Big then refers to big complexity rather than big volume.  Of course, valuable and complex datasets of this sort naturally tend to grow rapidly and so big data quickly becomes truly massive.

TAGGED:big data
Share This Article
Facebook Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

Flat editorial illustration: The article is about joining public web signals with internal business metrics so BI teams can inter
Alternative Data in BI: Bringing Public Web Data Into Your Dashboards
Big Data Business Intelligence Exclusive Web Analytics
Flat editorial illustration: A SaaS-editorial scene showing product usage, support patterns, and renewal feedback informing a fle
Using Product Usage Analytics and Support Data to Build a B2B Product Roadmap
Business Intelligence Infographic
From Hiring To Payroll: Managing HR Data Across Business Systems -- AI-generated illustration
From Hiring To Payroll: Managing HR Data Across Business Systems
Big Data Business Intelligence Infographic
Flat editorial illustration: The article is about automating the collection, preparation, and delivery of marketing data across p
From Data to Dashboards: 5 Marketing Reporting Automation Software Picks
Analytics Business Intelligence Exclusive Marketing

Stay Connected

1.2KFollowersLike
33.7KFollowersFollow
222FollowersPin

You Might also Like

Big Data Changes The Future Of The Data Recovery Industry
Big DataExclusive

Big Data Changes The Future Of The Data Recovery Industry

5 Min Read
Big Data is Transforming the Future of WordPress Hosting
Big DataExclusive

Big Data is Transforming the Future of WordPress Hosting

8 Min Read
Ways Big Data is Changing Capitalism for Centuries to Come
Big DataExclusivePolicy and GovernanceRisk Management

Ways Big Data is Changing Capitalism for Centuries to Come

6 Min Read
Fascinating Ways Big Data and DMPs Are Disrupting Online Publishing
Big DataExclusiveSoftware

Fascinating Ways Big Data and DMPs Are Disrupting Online Publishing

5 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

The Art of Conversation: Enhancing Chatbots with Advanced AI Prompts
The Art of Conversation: Enhancing Chatbots with Advanced AI Prompts
Chatbots
From Bolts to Bots: How AI Is Fortifying the Automotive Industry
From Bolts to Bots: How AI Is Fortifying the Automotive Industry
Artificial Intelligence

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-26 SmartData Collective. All Rights Reserved.
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?