By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData Collective
  • Analytics
    AnalyticsShow More
    construction analytics
    5 Benefits of Analytics to Manage Commercial Construction
    5 Min Read
    benefits of data analytics for financial industry
    Fascinating Changes Data Analytics Brings to Finance
    7 Min Read
    analyzing big data for its quality and value
    Use this Strategic Approach to Maximize Your Data’s Value
    6 Min Read
    data-driven seo for product pages
    6 Tips for Using Data Analytics for Product Page SEO
    11 Min Read
    big data analytics in business
    5 Ways to Utilize Data Analytics to Grow Your Business
    6 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: In a Petabyte Age, Is Understanding Passé?
Share
Notification Show More
Latest News
cloud-centric companies using network relocation
Cloud-Centric Companies Discover Benefits & Pitfalls of Network Relocation
Cloud Computing
construction analytics
5 Benefits of Analytics to Manage Commercial Construction
Analytics
database compliance guide
Four Strategies For Effective Database Compliance
Data Management
Digital Security From Weaponized AI
Fortifying Enterprise Digital Security Against Hackers Weaponizing AI
Security
DevOps on cloud
Optimizing Cost with DevOps on the Cloud
Development
Aa
SmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Analytics > Predictive Analytics > In a Petabyte Age, Is Understanding Passé?
Business IntelligencePredictive Analytics

In a Petabyte Age, Is Understanding Passé?

PaulBarsch1
Last updated: 2009/06/18 at 9:21 PM
PaulBarsch1
7 Min Read
SHARE
- Advertisement -
data imageAnalysts have estimated that the volume of data in enterprises of all sizes is doubling every two to three years. With the deluge of data, some companies are finding it makes more sense to discover and act upon patterns (i.e. customers who buy item X also buy item Y), rather than dig deeper and search for causation. In an age of cloud computing and “big data”—where correlation is often sufficient to gain business results—are we losing our thirst for knowledge and understanding?

Chris Anderson, editor of Wired Magazine and author of “The Long Tail” penned a provocative article in the July 2008 issue titled, “The End of Theory: The Data Deluge Makes the Scientific Method Obsolete.”

- Advertisement -

Mr. Anderson makes the claim that in “the Petabyte Age”, it’s more important for companies (and the marketers within them) to identify and act upon correlation first, and worry about context later.

For instance he writes, “Google’s founding philosophy is that we don’t know why this page is better than that one: if the statistics of incoming links say it is, that’s good enough. No semantic or causal analysis is required.”

More Read

Digital Security From Weaponized AI

Fortifying Enterprise Digital Security Against Hackers Weaponizing AI

10 Ways How Artificial Intelligence Is Changing the Content Writing Landscape
How IoT Can Be Connected to Business Intelligence
Amazing AI-based Image Upscaler From VanceAI
Actionable Tips To Set Up an AI Tech Venture Abroad

And dismissing many of the sciences that attempt to bring us understanding of …

data imageAnalysts have estimated that the volume of data in enterprises of all sizes is doubling every two to three years. With the deluge of data, some companies are finding it makes more sense to discover and act upon patterns (i.e. customers who buy item X also buy item Y), rather than dig deeper and search for causation. In an age of cloud computing and “big data”—where correlation is often sufficient to gain business results—are we losing our thirst for knowledge and understanding?

- Advertisement -

Chris Anderson, editor of Wired Magazine and author of “The Long Tail” penned a provocative article in the July 2008 issue titled, “The End of Theory: The Data Deluge Makes the Scientific Method Obsolete.”

Mr. Anderson makes the claim that in “the Petabyte Age”, it’s more important for companies (and the marketers within them) to identify and act upon correlation first, and worry about context later.

For instance he writes, “Google’s founding philosophy is that we don’t know why this page is better than that one: if the statistics of incoming links say it is, that’s good enough. No semantic or causal analysis is required.”

And dismissing many of the sciences that attempt to bring us understanding of the world around us, Mr. Anderson notes, “Who knows why people do what they do? The point is that they do it and we can track and measure it with unprecedented fidelity. With enough data, the numbers speak for themselves.”

As companies collect more data about their customers, competitors and macro/micro environments, Mr. Anderson makes the claim that our approach to science (hypothesize, model, test) is becoming obsolete.

- Advertisement -

Science looks for causation. Scientists hypothesize as to why something works/reacts/behaves like it does and then attempt to build a model to represent reality. The goal is to use the model to test and learn, thereby gaining an understanding of a particular phenomenon.

Modeling is not only confined to the realm of physicists and quants. In the field of marketing, managers often work alongside statisticians, to build models that help predict customer proclivities such as items they might like to buy, or identifying customers who might churn to a competitor. Models can be tested and refined and with enough tweaking, models generally get more accurate over time. The output of those models can be used to piece together more complete customer profiles.

However, in the Petabyte Age, Mr. Anderson claims, “There is now a better way. Petabytes allow us to say correlation is enough. We can stop looking at the models.”

Let’s suppose Mr. Anderson is on to something. Mr. Anderson makes the claim that since modeling is often a poor representation of reality and tends to over-simplify things, we’re much better off with less explanation and more action based on the identification of correlation.

In a sense, he says, in an age of massive data, we’re better off with fewer discoveries of knowledge and understanding.

- Advertisement -

I’m not sure I agree.

Peter Atkins, author of “Galileo’s Finger”, says it much better than I can: “With the rise of the computer and its ability to handle huge numerical calculations of the greatest intensity, we are seeing a shift from analysis to numerical computation. (This is dangerous because) resorting to numerical solution can distance us from understanding.”

It is true that building models is an imperfect science, and cannot in all instances be 100% accurate—it is after all just a model!

That said, models help us test our assumptions and verify our forecasts. They help us test what we think we know, with the ultimate goal of improving our decision making in real-life situations where marketing budgets and return on investment are on the line. Modeling helps us piece the world together, and look beyond the patterns towards discovering “why” things happen as they do. And models help us transform reams of raw data into intelligence thereby helping us predict outcomes with greater accuracy.

While focusing on correlation can help us make better decisions to a degree, philosophically I am concerned with marketers potentially losing our interest in truly understanding what makes our customers, prospects and partners tick. For me at least, I want to know more than “my customers do this, or they do that”; I want to know “why”!

- Advertisement -

In all fairness to Mr. Anderson, every coin has two sides. There is a fine line between too much pontification and too little action. Indeed, there are many instances where it doesn’t make sense to dig deeper in understanding—where it doesn’t matter “why”, only that a given solution produces results.

Link to original post

PaulBarsch1 June 18, 2009
Share this Article
Facebook Twitter Pinterest LinkedIn
Share
- Advertisement -

Follow us on Facebook

Latest News

cloud-centric companies using network relocation
Cloud-Centric Companies Discover Benefits & Pitfalls of Network Relocation
Cloud Computing
construction analytics
5 Benefits of Analytics to Manage Commercial Construction
Analytics
database compliance guide
Four Strategies For Effective Database Compliance
Data Management
Digital Security From Weaponized AI
Fortifying Enterprise Digital Security Against Hackers Weaponizing AI
Security

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

Digital Security From Weaponized AI
Security

Fortifying Enterprise Digital Security Against Hackers Weaponizing AI

11 Min Read
AI-powered content writing tools
Artificial Intelligence

10 Ways How Artificial Intelligence Is Changing the Content Writing Landscape

8 Min Read
internet of things and business intelligence
Internet of Things

How IoT Can Be Connected to Business Intelligence

6 Min Read
ai improves image upscaling
Artificial Intelligence

Amazing AI-based Image Upscaler From VanceAI

11 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

AI and chatbots
Chatbots and SEO: How Can Chatbots Improve Your SEO Ranking?
Artificial Intelligence Chatbots Exclusive
AI chatbots
AI Chatbots Can Help Retailers Convert Live Broadcast Viewers into Sales!
Chatbots

Quick Link

  • About
  • Contact
  • Privacy
Follow US

© 2008-23 SmartData Collective. All Rights Reserved.

Removed from reading list

Undo
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?