Cookies help us display personalized product recommendations and ensure you have great shopping experience.

By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    data analytics and truck accident claims
    How Data Analytics Reduces Truck Accidents and Speeds Up Claims
    7 Min Read
    predictive analytics for interior designers
    Interior Designers Boost Profits with Predictive Analytics
    8 Min Read
    image fx (67)
    Improving LinkedIn Ad Strategies with Data Analytics
    9 Min Read
    big data and remote work
    Data Helps Speech-Language Pathologists Deliver Better Results
    6 Min Read
    data driven insights
    How Data-Driven Insights Are Addressing Gaps in Patient Communication and Equity
    8 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: The Data Curves
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Big Data > Data Warehousing > The Data Curves
CommentaryData Warehousing

The Data Curves

MIKE20
MIKE20
4 Min Read
SHARE

Supply of -– and demand for –- data have never been higher. Executives often say, “I have all of the data that I can handle. I just need more information.”

Contents
Opposing ForcesSimon SaysFeedback

Consider just the data generated from mobile devices. From a recent ReadWriteWeb piece:

Supply of -– and demand for –- data have never been higher. Executives often say, “I have all of the data that I can handle. I just need more information.”

More Read

Kalido Directs Data Governance
Analyzing Logs and More – A Big Data Architecture
How Much Is Data Center Downtime Costing You?
Good Data Warehouse DBAs are Hard to Find
Eli Lilly’s Dave Powers talks compellingly about how the…

Consider just the data generated from mobile devices. From a recent ReadWriteWeb piece:

Worldwide mobile data traffic is due to increase 26-fold to 75 exabytes annually, says networking giant Cisco in its latest report, the Cisco Visual Networking Index Global Mobile Data Traffic Forecast for 2010 to 2015. To put that in perspective, that’s the equivalent of 19 billion DVDs, 536 quadrillion SMS text messages or 75 times the amount of global Internet IP data (fixed and mobile data) in the year 2000.

In a word, wow. At least there is some good news. Storing data has never been cheaper, a sign that shows no signs of abating. Consider that Amazon just announced its new AWS pricing. Outbound data transfer price for US-Standard, US-West and Europe regions are as follows:

  • $0.000 – first 1 GB / month data transfer out
  • $0.150 per GB – up to 10 TB / month data transfer out
  • $0.110 per GB – next 40 TB / month data transfer out
  • $0.090 per GB – next 100 TB / month data transfer out
  • $0.080 per GB – data transfer out / month over 150 TB

So, if I have 9 terabytes of data, outbound transfer costs will amount to $135 (USD)/month.

Opposing Forces

In other words, the amount of data out there is growing exponentially while its cost is dropping in a similar manner. This is crudely represented in my simple chart below:

chart

So, every day, we are generating more data  and storing it becomes cheaper. But do we need to store all of that data? The answer is usually no. This, of course, begs the obvious question: Which data do we need to keep?

And there’s the rub. I am of the opinion that it’s better to have it and not need it than need it and not have it. What’s more, keeping the data in one central repository is generally the way to go. Among the benefits of a “master data” strategy is the minimization of those pesky duplicate records that vex end-users trying to make business decisions based upon extra, inaccurate, or incomplete information.

What’s more, reporting is vastly simplified. To be sure, sophisticated reporting tools allow developers and extremely technical folks to pull data from myriad sources, tables and data models. However, performance tends to suffer and many non-technical users aren’t exactly proficient at writing complex SQL statements. This maximizes the need for IT to be involved when, in an ideal world, they should not have to “bless” each reporting request.

Simon Says

It’s never easy to determine which data is essential, which is nice-to-have, and which is no longer relevant. To boot, ask ten different users in an organization and you’re likely to receive ten different answers. Still, it behooves most organizations to routinely ask if they are storing the right information in the right manners. After all, needs are hardly static, especially these days.

Data storage is anything but a “set it and forget it” type of thing. Ignore changing requirements at your own peril.

Feedback

What say you?

Share This Article
Facebook Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

data analytics and truck accident claims
How Data Analytics Reduces Truck Accidents and Speeds Up Claims
Analytics Big Data Exclusive
predictive analytics for interior designers
Interior Designers Boost Profits with Predictive Analytics
Analytics Exclusive Predictive Analytics
big data and cybercrime
Stopping Lateral Movement in a Data-Heavy, Edge-First World
Big Data Exclusive
AI and data mining
What the Rise of AI Web Scrapers Means for Data Teams
Artificial Intelligence Big Data Exclusive

Stay Connected

1.2kFollowersLike
33.7kFollowersFollow
222FollowersPin

You Might also Like

100 Petabytes of Data in Poop?

6 Min Read

Hadoop pushes, pulls Big Data analytics into mainstream (Part Two)

8 Min Read

72% of People Aren’t Familiar with Hosted VoIP

4 Min Read

As we witness our society racing ahead with…

2 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

ai is improving the safety of cars
From Bolts to Bots: How AI Is Fortifying the Automotive Industry
Artificial Intelligence
ai chatbot
The Art of Conversation: Enhancing Chatbots with Advanced AI Prompts
Chatbots

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-25 SmartData Collective. All Rights Reserved.
Go to mobile version
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?