By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData CollectiveSmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    data-driven white label SEO
    Does Data Mining Really Help with White Label SEO?
    7 Min Read
    marketing analytics for hardware vendors
    IT Hardware Startups Turn to Data Analytics for Market Research
    9 Min Read
    big data and digital signage
    The Power of Big Data and Analytics in Digital Signage
    5 Min Read
    data analytics investing
    Data Analytics Boosts ROI of Investment Trusts
    9 Min Read
    football data collection and analytics
    Unleashing Victory: How Data Collection Is Revolutionizing Football Performance Analysis!
    4 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: Integrating NoSQL in the Data Warehouse
Share
Notification Show More
Aa
SmartData CollectiveSmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Big Data > Data Warehousing > Integrating NoSQL in the Data Warehouse
Data WarehousingUnstructured Data

Integrating NoSQL in the Data Warehouse

Barry Devlin
Last updated: 2011/01/21 at 4:03 PM
Barry Devlin
5 Min Read
SHARE

Just putting NoSQL in the title of a post on B-eye-Network might raise a few hackles 😉 but the growing popularity of the term and vaguely related phrases like big data, Hadoop and distributed file systems brings the topic regularly to the fore these days.  I’m often asked by BI practitioners: what is NoSQL and what can we do about it?

Just putting NoSQL in the title of a post on B-eye-Network might raise a few hackles 😉 but the growing popularity of the term and vaguely related phrases like big data, Hadoop and distributed file systems brings the topic regularly to the fore these days.  I’m often asked by BI practitioners: what is NoSQL and what can we do about it?

Broadly speaking, NoSQL is a rather loose term that groups together databases (and sometimes non-databases!) that do not use the relational model as a foundation.  And, like anything that is defined by what it’s not, NoSQL ends up being on one hand a broad church and on the other a focal point for those who strongly resist the opposite view.  NoSQL is thus claimed by some not to be anti-SQL, and said to stand for “not only SQL”.  But, let’s avoid this particular minefield and focus on the broad church of data stores that gather together under the NoSQL banner.

David Bessemer, CTO of Composite Software, gives a nice list in his “Data Virtualization and NoSQL Data Stores” article: (1) Tabular/Columnar Data Stores, (2) Document Stores, (3) Graph Databases, (4) Key/Value Stores, (5) Object and Multi-value Databases and (6) Miscellaneous Sources.  He then discusses how (1) and (4), together with XML document stores–a subset of (2)–can be integrated using virtualization tools such as Composite.

More Read

RDMBS databases

Beyond RDBMS: Databases for Modern Applications

In Big Data Endeavors, Don’t Neglect Softer Business Skills
Big Data’s Journey: From Big and Clumsy to Small and Cost-Effective
Some NoSQL Myths
Riptano for Cassandra

There is another school of thought that favors importing such data (particularly textual data) into the data warehouse environment, either by first extracting keywords from it via text analytics or by converting it to XML or other “relational-friendly” formats.  In my view, there is a significant problem with this approach; namely that the volumes of data are so large and their rate of change so fast in many cases, that traditional ETL and Data Warehouse infrastructures will struggle to manage.  The virtualization approach thus makes more sense as the mass access mechanism for such big data.

But, it’s also noticeable that Bessemer covers only 2.5 of his 6 classes in detail, saying that they are “particularly suited for the data virtualization platform”.  So, what about the others?

In my May 2010 white paper, “Beyond the Data Warehouse: A Unified Information Store for Data and Content“, sponsored by Attivio, I addressed this topic in some depth.  BI professionals need to look to what is emerging in the world of content management to see that soft information (also known by the oxymoronic term “unstructured information”) is increasingly being analyzed and categorized by content management tools to extract business meaning and value on the fly, without needing to be brought into the data warehouse.  What’s needed now is for content management and BI tool vendors to create the mechanism to join these two environments and create a common set of metadata that bridges the two.

This is also a form of virtualization, but the magic resides in the joint metadata.  Depending on your history and preferences, you can see this as an extension of the data warehouse to include soft information or an expansion of content management into relational data.  But, whatever you choose, the key point is to avoid duplicating NoSQL data stores into the data warehouse.

I’ll be speaking at O’Reilly Media’s big data oriented Strata Conference – Making Data Work – 1-3 February in Santa Clara, California. A keynote, The Heat Death of the Data Warehouse, Thursday, 3 February, 9:25am and an Exec Summit session, The Data-driven Business and Other Lessons from History, Tuesday, 1 February, 9:45am.  O’Reilly Media are offering a 25% discount code for readers, followers, and friends on conference registration:  str11fsd.  

TAGGED: nosql
Barry Devlin January 21, 2011
Share This Article
Facebook Twitter Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

IoT Cybersecurity
4 Common Misconceptions Surrounding IoT Cybersecurity Compliance
Internet of Things
iot and cloud technology
IoT And Cloud Integration is the Future!
Internet of Things
ai in marketing
4 Ways AI Can Improve Your Marketing Strategy
Artificial Intelligence
data security unveiled
Data Security Unveiled: Protecting Your Information in a Connected World
Security

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

RDMBS databases
Big DataBusiness IntelligenceExclusive

Beyond RDBMS: Databases for Modern Applications

8 Min Read
big data
AnalyticsBig DataBusiness IntelligenceCommentaryCulture/LeadershipData WarehousingExclusive

In Big Data Endeavors, Don’t Neglect Softer Business Skills

5 Min Read
big data
Big DataBusiness IntelligenceSoftware

Big Data’s Journey: From Big and Clumsy to Small and Cost-Effective

4 Min Read

Some NoSQL Myths

2 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

AI chatbots
AI Chatbots Can Help Retailers Convert Live Broadcast Viewers into Sales!
Chatbots
giveaway chatbots
How To Get An Award Winning Giveaway Bot
Big Data Chatbots Exclusive

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?