We use cookies, including third-party cookies from Google to serve personalized ads through AdSense, to operate this site and understand how it is used. By continuing to browse, you accept this use. See our Privacy Policy and Terms of Use for details, including how to opt out of personalized advertising.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    Flat editorial illustration: The article is about joining public web signals with internal business metrics so BI teams can inter
    Alternative Data in BI: Bringing Public Web Data Into Your Dashboards
    23 Min Read
    Flat editorial illustration: The article is about automating the collection, preparation, and delivery of marketing data across p
    From Data to Dashboards: 5 Marketing Reporting Automation Software Picks
    12 Min Read
    Flat editorial illustration: The article contrasts passive follower totals with a behavioral path from attention and trust to cli
    Creator Analytics Beyond Follower Count: Seven Metrics That Measure Audience Behavior
    11 Min Read
    Flat editorial illustration: The article connects AI and marketing analytics with managerial judgment grounded in consumer behavi
    How an Online MBA in Marketing Connects AI and Data Analytics
    11 Min Read
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers -- AI-generated illustration
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers
    7 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: Are You Making the Most of Your Dark Data?
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Big Data > Are You Making the Most of Your Dark Data?
Big Data

Are You Making the Most of Your Dark Data?

Timo Elliott
Timo Elliott
3 Min Read
Are You Making the Most of Your Dark Data?
Illustration generated with FLUX.2 [klein 4B] via Cloudflare Workers AI.
SHARE

Dark Data in the Cellar

My favorite definition of ‘Dark Data’ comes from Matt Aslett of 451 Research:

“Data that was previously ignored because of technology limitations.”

Dark Data in the Cellar

My favorite definition of ‘Dark Data’ comes from Matt Aslett of 451 Research:

“Data that was previously ignored because of technology limitations.”

I’ve recently talked to several big, traditional organizations that are finding lots of dark data down in the cellar.

There’s obviously the unstructured* data that companies have gathered but struggled to analyze in the past. Technologies like Hadoop make it much easier to get useful data from sources such as legal documents, the comments sections of customer surveys, medical research studies, social data, web logs, and many others.

But there’s also a lot of ‘dark’ structured data. It turns out that the number one use of Hadoop in most organizations today is analyzing old data that was previously too costly to process. In addition to providing known value, this data provides an easy, low-risk pilot project for getting to know Hadoop. The structures are well-known, so getting the data out in a useable format doesn’t require complex data exploration or new scripting skills.

Sometimes you may not realize that you have useful data. My favorite example of this is a project at Copenhagen Airport, where a team derived an amazing amount of useful information by crunching the data in the log files of the wifi routers scattered around the airport.

Passengers smart phones “ping” the different routers as they walk through the terminals, even if they don’t actually connect to the network, and the team found they could track passenger movements and behavior to a reasonable level of precision. The data was used to determine facilities questions around typical passenger flows and choke points, but also to help answer more commercial questions such as “which is the most visited area of duty free?”

The technology barriers have tumbled down, and it’s a good time to get out your bucket and flashlight and go take a look in your corporate cellar!

* Hate that label? Get over it– you and everybody else know what we’re talking about.

TAGGED:dark data
Share This Article
Facebook Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

Flat editorial illustration: The article connects downstream report failures, data-quality gaps, and conflicting metric definitio
Signs Your Data Architecture Needs an Outside Review
Big Data Exclusive
Data-Driven Pultrusion: Best GFRP Rebar Machines & ROI
Data-Driven Pultrusion: Best GFRP Rebar Machines & ROI
Big Data Exclusive
Flat editorial illustration: The article contrasts tools for gathering fresh interviews with tools for analyzing existing qualita
10 AI Market Research Tools That Turn Qualitative Data into Insights
Artificial Intelligence Big Data Exclusive
Flat editorial illustration: The article compares proxy MCP servers by what an AI agent can do through them: manage proxy account
Proxy MCP Servers: 7 Options and What Each Lets an AI Agent Do
Artificial Intelligence Big Data Data Collection

Stay Connected

1.2KFollowersLike
33.7KFollowersFollow
222FollowersPin

You Might also Like

5 Ways Dark Data Is Changing Data Analytics
AnalyticsBig DataData ManagementExclusiveSecurity

5 Ways Dark Data Is Changing Data Analytics

7 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

ai chatbot
How AI Website Chatbots Improve Customer Support and Lead Generation
Chatbots Exclusive
How To Get An Award Winning Giveaway Bot
How To Get An Award Winning Giveaway Bot
Big Data Chatbots Exclusive

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-26 SmartData Collective. All Rights Reserved.
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?