By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    data Analytics instagram stories
    Data Analytics Helps Marketers Make the Most of Instagram Stories
    15 Min Read
    analyst,women,looking,at,kpi,data,on,computer,screen
    What to Know Before Recruiting an Analyst to Handle Company Data
    6 Min Read
    AI analytics
    AI-Based Analytics Are Changing the Future of Credit Cards
    6 Min Read
    data overload showing data analytics
    How Does Next-Gen SIEM Prevent Data Overload For Security Analysts?
    8 Min Read
    hire a marketing agency with a background in data analytics
    5 Reasons to Hire a Marketing Agency that Knows Data Analytics
    7 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: Digital Universe Study: Extracting Value from Chaos
Share
Notification Show More
Aa
SmartData CollectiveSmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Analytics > Digital Universe Study: Extracting Value from Chaos
AnalyticsBusiness Intelligence

Digital Universe Study: Extracting Value from Chaos

Barry Devlin
Last updated: 2011/07/27 at 3:32 PM
Barry Devlin
6 Min Read
SHARE

I’ve just been reading the 5th annual Digital Universe study from IDC, released by EMC last month.  This year’s study seems to have attracted less media attention than previous versions.  Perhaps we’ve grown blasé about the huge numbers of bytes involved – 1.8 ZB (zettabytes, or 1.8 trillion gigabytes) in 2011 – or perhaps the fact that the 2011 number is exactly the same as predicted in the previous study is not newsworthy.  However, the subtitle of this year’s study, “Extracting

I’ve just been reading the 5th annual Digital Universe study from IDC, released by EMC last month.  This year’s study seems to have attracted less media attention than previous versions.  Perhaps we’ve grown blasé about the huge numbers of bytes involved – 1.8 ZB (zettabytes, or 1.8 trillion gigabytes) in 2011 – or perhaps the fact that the 2011 number is exactly the same as predicted in the previous study is not newsworthy.  However, the subtitle of this year’s study, “Extracting Value from Chaos”, should place it close to the top of every BI strategist’s reading list.  Here, and in my next blog entry, are a few of the key takeaways, some of which have emerged in previous versions of the study, but all of which together reemphasize that business intelligence needs to undergo a radical rebirth over the next couple of years.

1.8 ZB is a big number, but consider that it’s also a rapidly growing number, more than doubling every two years.  That’s faster than Moore’s Law.  By 2015, we’re looking at 7.5-8 ZB.  More than 90% of this information is already soft (aka unstructured) and that percentage is growing.  Knowing that the vast majority of this data is generated by individuals and much of that consists of video, image and audio, you may ask: what does this matter to my internal BI environment?  The answer is: it matters a lot!  Because in that vast, swirling and ever-changing cosmic miasma of data there are hidden the occasional nuggets of valuable insight.  And whoever gets to them first – you or your competitors – will potentially gain significant business advantage.

With such volumes of information and such rapid growth, it is simply impossible to examine (never mind analyse) it manually.  This demands an automated approach.  Such tools are emerging – for example, facial recognition of photos on Facebook and elsewhere or IBM Watson’s extraction of Jeopardy answers from the contents of the Internet.  Conceptually, what such tools do is generate data about data, which, as we know and love in BI, means metadata.  According to IDC, metadata is growing at twice the rate of the digital universe as a whole.  That’s more than doubling every year!  

More Read

what cyber criminals can do with your metadata

What Cyber Criminals Can Do With Your Metadata

Big Data, Small Details: How Metadata Creates Security Risks
Metadata and the Baker/Baker Paradox
The Enterprise Brain
SAS Admin: Process Data Faster in RDBMS by Buffering the Data in Memory

So, while we may well ask what you’re doing about managing and storing soft information, an even more pressing question is what are you going to do about metadata?  Of course, the volumes of metadata are probably still relatively small (IDC hasn’t published an absolute value), but that growth rate means they will get large; fast.  And we currently have a much more limited infrastructure and weaker methodologies to handle metadata than we’ve created over the years for data.  Not to mention that the value to be found in the chaos can be discovered only through the lens of the metadata that characterizes the data itself.

For BI, this shift in focus from hard to soft information is only one of the changes we have to manage.  Another major change involves the nature and sources of the hard data itself.  There is a growing quantity of hard data collected from machine sensors as more and more of the physical world goes on the Web.  RFID readers are generating ever increasing volumes of data.  (According to VDC Research, nearly 4 billion RFID tags were sold in 2010, a 35% increase over the previous year.)  From electricity meters to automobiles, intelligent, connected devices are pumping out ever increasing volumes of data that is being used in a wide variety of new applications.  And almost all of these applications can be characterized as operational BI.  So, the move from traditional, tactical BI to the near real-time world of operational BI is accelerating, with all of the challenges that entails.

Next time, I’ll be looking at some of the implications of the changes in sourcing on security and privacy, as well as the interesting fact that although the stored digital universe is huge, the transient data volumes are a number of orders of magnitude higher.

TAGGED: metadata
Barry Devlin July 27, 2011
Share This Article
Facebook Twitter Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

smart home data
7 Mind-Blowing Ways Smart Homes Use Data to Save Your Money
Big Data
ai low code frameworks
AI Can Help Accelerate Development with Low-Code Frameworks
Artificial Intelligence
data Analytics instagram stories
Data Analytics Helps Marketers Make the Most of Instagram Stories
Analytics
data breaches
How Hospital Security Breaches Devastate Local Communities
Policy and Governance

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

what cyber criminals can do with your metadata
Best PracticesBig DataData ManagementExclusivePrivacyRisk ManagementSecurity

What Cyber Criminals Can Do With Your Metadata

8 Min Read
big data and meta data
Big DataSecurity

Big Data, Small Details: How Metadata Creates Security Risks

5 Min Read
Image
Big DataData Management

Metadata and the Baker/Baker Paradox

4 Min Read
data warehouse
AnalyticsBig DataBusiness IntelligenceData ManagementData WarehousingUnstructured Data

The Enterprise Brain

5 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

giveaway chatbots
How To Get An Award Winning Giveaway Bot
Big Data Chatbots Exclusive
AI and chatbots
Chatbots and SEO: How Can Chatbots Improve Your SEO Ranking?
Artificial Intelligence Chatbots Exclusive

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?