By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData Collective
  • Analytics
    AnalyticsShow More
    data analytics in sports industry
    Here’s How Data Analytics In Sports Is Changing The Game
    6 Min Read
    data analytics on nursing career
    Advances in Data Analytics Are Rapidly Transforming Nursing
    8 Min Read
    data analytics reveals the benefits of MBA
    Data Analytics Technology Proves Benefits of an MBA
    9 Min Read
    data-driven image seo
    Data Analytics Helps Marketers Substantially Boost Image SEO
    8 Min Read
    construction analytics
    5 Benefits of Analytics to Manage Commercial Construction
    5 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: General Purpose Sensemaking Systems and Information Colocation
Share
Notification Show More
Latest News
big data mac performance
Data-Driven Tips to Optimize the Speed of Macs
News
3 Ways AI Has Helped Marketers and Creative Professionals Streamline Workflows
3 Ways AI Has Helped Marketers and Creative Professionals Streamline Workflows
Artificial Intelligence
data analytics in sports industry
Here’s How Data Analytics In Sports Is Changing The Game
Big Data
data analytics on nursing career
Advances in Data Analytics Are Rapidly Transforming Nursing
Analytics
data analytics reveals the benefits of MBA
Data Analytics Technology Proves Benefits of an MBA
Analytics
Aa
SmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Big Data > Data Warehousing > General Purpose Sensemaking Systems and Information Colocation
Data Warehousing

General Purpose Sensemaking Systems and Information Colocation

JeffJonas
Last updated: 2010/11/22 at 3:22 PM
JeffJonas
5 Min Read
SHARE

General purpose sensemaking systems will colocate diverse data in the same data space.  Such an approach enables massively scalable, real-time, novel discovery over an ever changing observational space – without re-engineering.

General purpose sensemaking systems will colocate diverse data in the same data space.  Such an approach enables massively scalable, real-time, novel discovery over an ever changing observational space – without re-engineering.

This of course should be no surprise.  Ever since Von Neumann’s assertion that computer memory should be used for operating instructions and data – not two different memories for the different purposes – general purpose computing became possible.

More Read

What is Data Pipeline A detailed explaination

What is Data Pipeline? A Detailed Explanation

Understanding ETL Tools as a Data-Centric Organization
Differentiating Between Data Lakes and Data Warehouses
How Will The Cloud Impact Data Warehousing Technologies?
Big Data Is More Prevalent in Daily Life Than You Might Think

“Von Neumann machines differ in that they have a memory in which they store their operating instructions and data.  Such computers are more versatile in that they do not need to have their hardware reconfigured for each new program, but can simply be reprogrammed with new in-memory instructions; they also tend to be simpler to design, in that a relatively simple processor may keep state between successive computations to build up complex procedural results.  Most modern computers are von Neumann machines.”

~ Wikipedia: Computer data storage

Data structure governs function.  For example, a room full of DVD’s behaves one way and a SQL database behaves another.  Same holds true with enterprise operational systems: The human resources system uses one data model and the hotel reservation system another – each underlying data structure designed for each specific mission – and notably, of little use to anything else.

With information trapped in the tailored database schemas of systems of record, operational data stores, data warehouses and data marts, it is no wonder organizations continue to struggle to make sense of it all – despite decades of effort and innovation.

Performing some kind of federated search over all these disparate data sets just has not ever delivered.  In fact, federated search bites when it comes to sensemaking because the diverse data structures are incapable of supporting a sensemaking function.

If you want to be smart, you will want to jam the available, diverse, observational space into the same data structure and in as close to the same physical space as possible. 

Data is data.

When reference data, transactional data, and even user queries are colocated in the same data structures and is the same indexes as the extracted features from text, video, biometrics, and so on … something very exciting happens: data naturally finds data and context can accumulate.

By way of background: I first stumbled into the importance of data colocation back in 1993 when designing a surveillance system for the casinos in Vegas – a system that would help them keep the bad guys out.  After claiming I could build such a system in 90 days for $25k, I was forced to take some short-cuts.  Honestly, had the casino given me more time and been willing to pay more money I would have created a much more elaborate system containing a number of tailored database schemas (e.g., different structures for customers, employees, bad guys, vendors, stored user queries, etc.).  Given the time and money constraints, I had to make some compromises.  I decided to design one schema to support everything.  Each record in the system would then be designated with a role e.g., “Customer” or “Bad Guy.”  Long story short, when this general sensemaking system came on-line it started finding marketing hosts comping their roommates and lots of other unanticipated novel discovery.  So much novel discovery, it earned the name Non-Obvious Relationship Awareness or NORA, we got two rounds of funding, IBM bought my company to get its hands on the technology, and the rest is history.

Simply said, you have to have a brain (multi-purpose, general structure) to think (sense make).  Then with a brain, the smartest you are going to be is a function of what observations you have properly contextualized into that meat space between your ears.

JeffJonas November 22, 2010
Share this Article
Facebook Twitter Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

big data mac performance
Data-Driven Tips to Optimize the Speed of Macs
News
3 Ways AI Has Helped Marketers and Creative Professionals Streamline Workflows
3 Ways AI Has Helped Marketers and Creative Professionals Streamline Workflows
Artificial Intelligence
data analytics in sports industry
Here’s How Data Analytics In Sports Is Changing The Game
Big Data
data analytics on nursing career
Advances in Data Analytics Are Rapidly Transforming Nursing
Analytics

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

What is Data Pipeline A detailed explaination
Big Data

What is Data Pipeline? A Detailed Explanation

8 Min Read
etl for data-driven businesses
Big Data

Understanding ETL Tools as a Data-Centric Organization

8 Min Read
data lake vs data warehouse
Data Lake

Differentiating Between Data Lakes and Data Warehouses

7 Min Read
moving to the cloud
Big DataCloud ComputingData WarehousingExclusive

How Will The Cloud Impact Data Warehousing Technologies?

6 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

data-driven web design
5 Great Tips for Using Data Analytics for Website UX
Big Data
AI and chatbots
Chatbots and SEO: How Can Chatbots Improve Your SEO Ranking?
Artificial Intelligence Chatbots Exclusive

Quick Link

  • About
  • Contact
  • Privacy
Follow US

© 2008-23 SmartData Collective. All Rights Reserved.

Removed from reading list

Undo
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?