By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData CollectiveSmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    customer experience analytics
    Using Data Analysis to Improve and Verify the Customer Experience and Bad Reviews
    6 Min Read
    data analytics and CRO
    Data Analytics is Crucial for Website CRO
    9 Min Read
    analytics in digital marketing
    The Importance of Analytics in Digital Marketing
    8 Min Read
    benefits of investing in employee data
    6 Ways to Use Data to Improve Employee Productivity
    8 Min Read
    Jira and zendesk usage
    Jira Service Management vs Zendesk: What Are the Differences?
    6 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: Are Smart Data Lakes the Answer to Data Warehouses?
Share
Notification Show More
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Big Data > Data Warehousing > Are Smart Data Lakes the Answer to Data Warehouses?
Big DataData ManagementData WarehousingExclusive

Are Smart Data Lakes the Answer to Data Warehouses?

Sean Martin
Last updated: 2016/08/02 at 12:36 PM
Sean Martin
5 Min Read
Image
SHARE

A phenomenal shift has occurred over the past few years in the enterprise data world. The ubiquitous data warehouses – the foundation for business intelligence and data discovery for several decades – are now becoming obsolete due to the emergence of data lakes.

While both data warehouses and data lakes have their pros and cons, a new era of ‘smart’ data lakes based on semantic technology is emerging that can reduce the disadvantages of either, creating a clear path for the industry.

A phenomenal shift has occurred over the past few years in the enterprise data world. The ubiquitous data warehouses – the foundation for business intelligence and data discovery for several decades – are now becoming obsolete due to the emergence of data lakes.

While both data warehouses and data lakes have their pros and cons, a new era of ‘smart’ data lakes based on semantic technology is emerging that can reduce the disadvantages of either, creating a clear path for the industry.

Before the evolution of the data lake, data warehouses were the only reliable solution for producing analytical reports on enterprise data. The benefits of a well-implemented data warehouse include effective governance and security, high data quality, and consistent analytics performance over time.

The downside of a data warehouse is it requires traditional, complex tools, like those from Oracle, IBM and Microsoft that typically require expensive resources and infrastructure.  Extensive preparation is also needed by skilled IT analysts to set up the warehouse, and there is very little flexibility in adapting to the rapidly changing business landscape. Many organizations also view data warehouses negatively due to their high failure rates and ongoing costs.

Due to the decreasing costs of data storage, enterprises began turning to data lakes as an alternative in recent years. Data lakes serve as large repositories of structured and unstructured data that many in the industry hoped could be accessed to extract value relatively quickly using big data tools.

However, like all first-generation solutions, data lakes presented some concerns as well. While data lakes enable business analysts to quickly and efficiently query unstructured data, the skills required of these analysts are scarce, often requiring extensive training.  In addition, it’s difficult to have data lakes adhere to critical rules of data governance, such as maintaining the security, access control and integrity of enterprise data.

Because of these issues, interest in “smart” data lakes has risen in an attempt to secure the benefits of data warehouses and first-generation data lakes while reducing their negatives.

Smart data lakes use semantic graph query engines that link and contextualize huge volumes of diverse enterprise data to determine meaning and value.  The graph models enable independent data discovery, analytics and visualization capabilities by users across all entities and relationships in the vast data lake repository.

The graph models of smart data lakes also eliminate the need for the extensive preparation required of data warehouses and the training required for effective analysis in first-generation data lakes.  There is generally some upfront preparation still required by IT personnel for smart data lakes, but it pales in comparison to the scale of work required for data warehouses.  In addition, the relative ease of working with graph-based models opens the door to a variety of business users querying the data, leading to a ‘democratization’ of big data discovery and analysis.

Semantic technologies also help maintain the necessary data governance needed for the long-term sustainability of data lakes. Organizations can implement access to data in accordance with governance protocols by specifying who can and cannot view data elements.  With these model-driven governance and access controls, restrictions and permissions are as enforceable as if the data were siloed according to governance mandates.  This empowers users to ask questions while retaining trust that the answers stem from high-quality, secure data.

With key benefits of smart data lakes in the chart below, smart data lakes are well-positioned to displace data warehousing as the de facto means of storing data and facilitating analytics and discovery.

Image

Sean Martin August 2, 2016 August 2, 2016
Share This Article
Facebook Twitter Pinterest LinkedIn
Share
By Sean Martin
Follow:
Sean Martin has been on the leading edge of Internet technology innovation since the early nineties. His greatest strength has been the identification and pioneering of next generation software & networking technologies and techniques. Prior to founding Cambridge Semantics, the leading provider of smart data solutions driven by semantic web technology, he spent fifteen years with IBM Corporation where he was a founder and the technology visionary for the IBM Advanced Internet Technology group.He is a native of South Africa, has lived for extended periods in London, England and Edinburgh, Scotland, but now makes his home in Boston, Mass.

Follow us on Facebook

Latest News

ai can help with nurse burnout
Breakthroughs in AI Are Helping to Prevent Nurse Burnout
Artificial Intelligence Exclusive
AI in marketing
AI Can’t Replace Creativity When Crafting Digital Content
Artificial Intelligence
ai in furniture design
Top 5 AI-Driven Furniture Engineering Design Applications
Artificial Intelligence
data protection regulation
Benefits of Data Management Regulations for Consumers & Businesses
Data Management

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

ai can help with nurse burnout
Artificial IntelligenceExclusive

Breakthroughs in AI Are Helping to Prevent Nurse Burnout

9 Min Read
AI in marketing
Artificial Intelligence

AI Can’t Replace Creativity When Crafting Digital Content

6 Min Read
ai in furniture design
Artificial Intelligence

Top 5 AI-Driven Furniture Engineering Design Applications

11 Min Read
data protection regulation
Data Management

Benefits of Data Management Regulations for Consumers & Businesses

13 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

ai chatbot
The Art of Conversation: Enhancing Chatbots with Advanced AI Prompts
Chatbots
AI chatbots
AI Chatbots Can Help Retailers Convert Live Broadcast Viewers into Sales!
Chatbots

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?