By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    data Analytics instagram stories
    Data Analytics Helps Marketers Make the Most of Instagram Stories
    15 Min Read
    analyst,women,looking,at,kpi,data,on,computer,screen
    What to Know Before Recruiting an Analyst to Handle Company Data
    6 Min Read
    AI analytics
    AI-Based Analytics Are Changing the Future of Credit Cards
    6 Min Read
    data overload showing data analytics
    How Does Next-Gen SIEM Prevent Data Overload For Security Analysts?
    8 Min Read
    hire a marketing agency with a background in data analytics
    5 Reasons to Hire a Marketing Agency that Knows Data Analytics
    7 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: No, Hadoop Is Not Going To Replace Your Data Warehouse
Share
Notification Show More
Aa
SmartData CollectiveSmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Software > Hadoop > No, Hadoop Is Not Going To Replace Your Data Warehouse
HadoopSoftware

No, Hadoop Is Not Going To Replace Your Data Warehouse

Timo Elliott
Last updated: 2014/04/11 at 10:13 AM
Timo Elliott
8 Min Read
SHARE

The data says that Hadoop isn’t going to replace your enterprise data warehouse.

Contents
Hadoop Adds Rather Than SubtractsHadoop Adds Rather Than SubtractsEmbrace Technology Change Outside of Hadoop, TooBut isn’t Hadoop going to do all that?The Architecture of the FutureIt’s About The Applications of The Future

Hadoop Adds Rather Than Subtracts

According to Gartner’s latest surveys, the number of CIOs that think that Hadoop will replace their existing analytics infrastructure has plummeted over the last few years, and is now down to just 3%.

The data says that Hadoop isn’t going to replace your enterprise data warehouse.

Hadoop Adds Rather Than Subtracts

According to Gartner’s latest surveys, the number of CIOs that think that Hadoop will replace their existing analytics infrastructure has plummeted over the last few years, and is now down to just 3%.

More Read

What is Data Pipeline A detailed explaination

What is Data Pipeline? A Detailed Explanation

Differentiating Between Data Lakes and Data Warehouses
What is an Enterprise Data Warehouse?
Data Integration Ecosystem for Big Data and Analytics
What Will We Call Big Data in 2015?

gartner dw hadoop

Are these CIOs deluded? No, the fact that the numbers continue to drop are a good indication that they understand the technology — and its limitations.

Note this DOESN’T mean that enterprises don’t see value of Hadoop, or aren’t going to massively increase their investment in it. Quite the opposite. At this week’s Hadoop Summit in Amsterdam, I heard from large enterprises such as Deutsche Telekom, Centrica, EDF, HSBC, and ING bank that are making big strategic bets on Hadoop as a core data framework for the future.

But like the survey respondents, they believe that Hadoop ultimately “adds more than subtracts.” For example, Alasdair Anderson of HSBC gave great, concrete example of the cost and flexibility advantages of Hadoop in his presentation “Hadoop-economics, the Invisible Hand of Big Data.” But he also included a “Health Warning” to use Hadoop where it makes sense:

“There’s no relationship between the EDW and Hadoop right now — they are going to be complimentary. It’s NOT about rip and replace: we’re not going to get rid of RDBMS or MPP, but instead use the right tool for right job — and that will very much be driven by price.”

Ralph Kimball, one of the key data warehousing pioneers, echoed these sentiments in a recent CloudEra webinar. In a must-see presentation designed to explain Hadoop to data warehouse experts, he positively gushed over the new opportunities, but here’s what he had to say in the Q&A section:

“Here’s a question that made me laugh a little bit, but it’s a serious question: ‘Well does this mean that relational databases are going to die?’. I think that there was a sense, three or four years ago, that maybe this was all a giant zero sum game between Hadoop and relational databases, and that has simply gone away. Everyone has now realized that there’s a huge legacy value in relational databases for the purposes they are used for. Not only transaction processing, but for all the very focused, index-oriented queries on that kind of data, and that will continue in a very robust way forever. Hadoop, therefore, will present this alternative kind of environment for different types of analysis for different kinds of data, and the two of them will coexist. And they will call each other. There may be points at which the business user isn’t actually quite sure which one of them they are touching at any point of time.”

Embrace Technology Change Outside of Hadoop, Too

So Hadoop isn’t going to “replace” EDW. But how about that notion that EDWs are legacy, to be maintained but not used for new analytic applications? Will EDWs become “the mainframes of 21st century” as one Hadoop Summit attendee put it?

No. Ignoring the many advantages of Hadoop would be dumb. But it would be just as dumb to ignore the other revolutionary technology breakthroughs in the DW space. In particular, new in-memory processing opportunities have created a brand-new category that Gartner calls “hybrid transactional/analytic platforms” (HTAP):

“Hybrid transaction/analytical processing will empower application leaders to innovate via greater situation awareness and improved business agility. This will entail an upheaval in the established architectures, technologies and skills driven by use of in-memory computing technologies as enablers.”

image

HTAP isn’t just a way of speeding up existing applications and analytics. The simplicity of the architecture (data is stored just once), and the flexibility of the platform means it has a lower total cost of ownership that traditional disk-based platforms. This means it’s also rapidly becoming the default platform for new business application deployments, and the heart of new “real-time” applications.

But isn’t Hadoop going to do all that?

No. This was a gratingly prevalent view at at Hadoop Summit, but it’s firmly in the “wishful thinking” camp.

Yes, there are projects to make in-memory and ACID compliance part of the Hadoop Framework. Storm and Flume mean you can start using Hadoop with streaming data. YARN may turn into a “generic app provisioning system via Linux containers.”

Does this mean that you’ll be able to do more with Hadoop in the future? Yes. Is it going to be easier to make applications? Yes. Is forty years of business process and data warehousing technology and expertise going be obsolete any time soon? No!

One small example, from Alan Gates, one of the co-founders of Hortonworks, presenting the new ACID features in the next version of Hive, answering a question about whether this would enable OLTP support:

“We’re not aiming at that, it’s not what HIVE is good at, we don’t think it makes any sense, and it would be a total fail if you did that.”

There are other projects for adding transactions to Hadoop, but integration with the corporate world will require support for standards like SQL, and the irony is that it will be a big challenge for Hadoop to support even 22yr-old SQL-92. As analyst Merv Adrian put it in a recent Gartner conference presentation: “What is remarkable is that Hadoop does SQL. Just don’t expect it to do it well”. (More in Gartner’s report: “choosing your SQL access strategy for Hadoop”)

The Architecture of the Future

The right answer for the future is the boring, pragmatic one that big data organizations like Hortonworks emphasize in their “modern data architecture” diagram. To make the most of your data, you will need all these proven and new technologies working seamlessly together.

hadoop modern architecture

It’s About The Applications of The Future

The real opportunity of new technologies like Hadoop and In-Memory processing is to enable new, more flexible, analytics-focused, actionable applications. And that takes much more than just a platform. Organizations want best-practice business applications capable of analyzing big data and putting it in the hands of people on the front line of your organization that need it, via cloud and mobile devices. And they want a vibrant ecosystem capable of helping the organization make the best use of those applications.

For more information about what that might look like, take a look at SAP’s big data solutions.

TAGGED: data warehouse
Timo Elliott April 11, 2014
Share This Article
Facebook Twitter Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

ai low code frameworks
AI Can Help Accelerate Development with Low-Code Frameworks
Artificial Intelligence
data Analytics instagram stories
Data Analytics Helps Marketers Make the Most of Instagram Stories
Analytics
data breaches
How Hospital Security Breaches Devastate Local Communities
Policy and Governance
analyst,women,looking,at,kpi,data,on,computer,screen
What to Know Before Recruiting an Analyst to Handle Company Data
Analytics

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

What is Data Pipeline A detailed explaination
Big Data

What is Data Pipeline? A Detailed Explanation

8 Min Read
data lake vs data warehouse
Data Lake

Differentiating Between Data Lakes and Data Warehouses

7 Min Read
Data Warehouse
Big DataData Warehousing

What is an Enterprise Data Warehouse?

6 Min Read
Data Integration Architecture
AnalyticsBig DataData ManagementExclusiveIT

Data Integration Ecosystem for Big Data and Analytics

8 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

data-driven web design
5 Great Tips for Using Data Analytics for Website UX
Big Data
ai in ecommerce
Artificial Intelligence for eCommerce: A Closer Look
Artificial Intelligence

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?