We use cookies, including third-party cookies from Google to serve personalized ads through AdSense, to operate this site and understand how it is used. By continuing to browse, you accept this use. See our Privacy Policy and Terms of Use for details, including how to opt out of personalized advertising.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    Flat editorial illustration: The article is about joining public web signals with internal business metrics so BI teams can inter
    Alternative Data in BI: Bringing Public Web Data Into Your Dashboards
    23 Min Read
    Flat editorial illustration: The article is about automating the collection, preparation, and delivery of marketing data across p
    From Data to Dashboards: 5 Marketing Reporting Automation Software Picks
    12 Min Read
    Flat editorial illustration: The article contrasts passive follower totals with a behavioral path from attention and trust to cli
    Creator Analytics Beyond Follower Count: Seven Metrics That Measure Audience Behavior
    11 Min Read
    Flat editorial illustration: The article connects AI and marketing analytics with managerial judgment grounded in consumer behavi
    How an Online MBA in Marketing Connects AI and Data Analytics
    11 Min Read
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers -- AI-generated illustration
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers
    7 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: Hygienic Hadoop Data Lakes Not Just Happenstance
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Software > Hadoop > Hygienic Hadoop Data Lakes Not Just Happenstance
CommentaryData ManagementExclusiveHadoopOpen SourcePolicy and Governance

Hygienic Hadoop Data Lakes Not Just Happenstance

paulbarsch
paulbarsch
4 Min Read
Hygienic Hadoop Data Lakes Not Just Happenstance
Illustration generated with Qwen Image.
SHARE

It is often thought that Apache® Hadoop based data lakes are a potential panacea to thorny data management issues long germane to relational databases. After all, the (mistaken) belief goes, you can simply dump all your data into Hadoop’s file system, and via schema on read magic, your desired result sets will appear with very little effort. However, data management—even for Hadoop—isn’t going away and in fact, probably never will.

If you’ve never read Tyler Brûlé’s columns in the Financial Times, you’re really missing something. Mr. Brûlé’s column is a Sunday morning staple where he comments on design, style, business, travel and more.  Even better, Mr. Brûlé was recently paired with the FT’s Lucy Kellaway in an article where they discussed Mr. Brûlé’s obsession with cleanliness, order, and aesthetics.

Reading along, I found some interesting parallels with Mr. Brûlé’s observations on office clutter, and relational database design practices.

For example, Mr. Brûlé despises anarchy. In the article he pointed to a staffer’s empty Evian water bottle on a desk, complaining that such items take away from his emphasis on office décor. And most certainly, Mr. Brûlé does not like jackets on the back of chairs, and anything else that takes away from the intended design and decoration of the office.  Why? “There needs to be a rule of law, or else where does it end?” he says. Otherwise “people will come in with wheelie suitcases, or with plastic hangers and dry cleaning.”

More Read

How To Solve The Data Management Challenge Of Self-Driving Cars
How To Solve The Data Management Challenge Of Self-Driving Cars
4 IT Management Best Practices Data-Driven Businesses Must Practice
VPNs Are Essential Data Protection Tools for Home Offices
Is Big Data Creating A Competitive Edge For Small Businesses?
Internet2: Big Pipes for Big Data

While some may find all this attention to detail slightly amusing, Mr. Brûlé does not. And neither does your company database administrator (DBA). That’s because in order to provide accurate reports, BI visualizations and powerful analytics, there are significant efforts that must take place to identify, model, transform, curate and secure data in a relational database. In essence, all the work to make your data look as clean, ordered and useful as Mr. Brûlé’s office is an ongoing process handled by your data stewards and DBAs.

Now let’s get back to Hadoop. Almost no one who works with Hadoop on a daily basis would suggest that data can simply be dumped into Hadoop’s file system and be of high value to rank and file business users.

Want to store sensitive data in your data lake? You’ll most certainly be doubling down on your efforts to lockup key data, especially since Hadoop security is evolving. In addition to data security, you’ll still have to contend with metadata management, architecture and design, and governance in Hadoop. Indeed, none of these data management issues are going away if you’re planning on allowing Hadoop to serve as a true lake or “hub” for all your organization’s data.

Are you just starting out with your Hadoop data lake and not quite there yet in terms of clear and accepted data management processes? It probably seems like a gargantuan task at first, but as the old yarn goes, you eat the elephant one bite at time. Or in the words of the esteemed Tyler Brûlé; “It’s a daily effort to adhere to set standards. (But) you need to aspire to something.”  Even if that “something” is a well governed, managed and secured Hadoop based data lake.

TAGGED:risky business
Share This Article
Facebook Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

b2b payment data tells supplier about its customers
What B2B Payment Data Tells a Supplier About Its Customers
Business Intelligence
Flat editorial illustration: The article connects downstream report failures, data-quality gaps, and conflicting metric definitio
Signs Your Data Architecture Needs an Outside Review
Big Data Exclusive
Data-Driven Pultrusion: Best GFRP Rebar Machines & ROI
Data-Driven Pultrusion: Best GFRP Rebar Machines & ROI
Big Data Exclusive
Flat editorial illustration: The article contrasts tools for gathering fresh interviews with tools for analyzing existing qualita
10 AI Market Research Tools That Turn Qualitative Data into Insights
Artificial Intelligence Big Data Exclusive

Stay Connected

1.2KFollowersLike
33.7KFollowersFollow
222FollowersPin

You Might also Like

Less Dogma Equals Better Decision Making
CommentaryData ManagementData WarehousingDecision ManagementExclusiveMobilityPolicy and Governance

Less Dogma Equals Better Decision Making

5 Min Read
Changing Your Mind About Big Data Isn’t Dumb
Best PracticesBig DataCloud ComputingCommentaryCulture/LeadershipData WarehousingExclusiveHadoopITUnstructured Data

Changing Your Mind About Big Data Isn’t Dumb

6 Min Read
Science Needs to Be Less Certain
AnalyticsBig DataCommentaryData VisualizationData WarehousingExclusiveModelingPredictive AnalyticsRisk Management

Science Needs to Be Less Certain

5 Min Read
4 Business Risks That Might Prevent Big Data ROI
Big DataBusiness IntelligenceRisk Management

4 Business Risks That Might Prevent Big Data ROI

5 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

AI chatbots
AI Chatbots Can Help Retailers Convert Live Broadcast Viewers into Sales!
Chatbots
The Art of Conversation: Enhancing Chatbots with Advanced AI Prompts
The Art of Conversation: Enhancing Chatbots with Advanced AI Prompts
Chatbots

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-26 SmartData Collective. All Rights Reserved.
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?