We use cookies, including third-party cookies from Google to serve personalized ads through AdSense, to operate this site and understand how it is used. By continuing to browse, you accept this use. See our Privacy Policy and Terms of Use for details, including how to opt out of personalized advertising.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    Flat editorial illustration: The article is about joining public web signals with internal business metrics so BI teams can inter
    Alternative Data in BI: Bringing Public Web Data Into Your Dashboards
    23 Min Read
    Flat editorial illustration: The article is about automating the collection, preparation, and delivery of marketing data across p
    From Data to Dashboards: 5 Marketing Reporting Automation Software Picks
    12 Min Read
    Flat editorial illustration: The article contrasts passive follower totals with a behavioral path from attention and trust to cli
    Creator Analytics Beyond Follower Count: Seven Metrics That Measure Audience Behavior
    11 Min Read
    Flat editorial illustration: The article connects AI and marketing analytics with managerial judgment grounded in consumer behavi
    How an Online MBA in Marketing Connects AI and Data Analytics
    11 Min Read
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers -- AI-generated illustration
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers
    7 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: The Emerging Big Data Ecosystem
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Business Intelligence > Knowledge Management > The Emerging Big Data Ecosystem
Big DataKnowledge ManagementNew Products

The Emerging Big Data Ecosystem

Barry Devlin
Barry Devlin
0 Min Read
The Emerging Big Data Ecosystem
Illustration generated with FLUX.2 [Klein 9B] via local ComfyUI.
SHARE

Slowly but surely, big data is becoming mainstream.  Of course, if you listened only to the hype from analysts and vendors, you might think this was already the case.  I suspect it’s more like teenage sex, more talked about than actually happening.  But, seems like we’re about to move into roaring twenties.

I had the pleasure to be invited as the external expert speaker at IBM’s PureData launch in Boston this week.  In a theatrical, dry-ice moment, IBM rolled out one of their new PureData machines between the previously available PureFlex and PureApplication models.  However, for me, the launch carried a much more complex and, indeed, subtle message than “here’s our new, bright and shiny hardware”.  Rather, it played on a set of messages that is gradually moving big data from a specialized and largely standalone concept to an all-embracing, new ecosystem that includes all data and the multifarious ways business needs to use it.

Despite long-running laments to the contrary, IT has had it easy when it comes to data management and governance.  Before you flame me, please read at least the rest of this paragraph.  Since the earliest days of general-purpose business computing in the 1960s, we’ve worked with a highly modeled and carefully designed representation of reality.  Basically, we’ve taken the messy, incoherent record of what really happens in the real word and hammered it into relational (and previously popular hierarchical or network) databases.  To do so, we’ve worked with highly simplified models of the world.  These simplifications range from grossly wrong (all addresses must include a 5-digit zip-code–yes, there are still a few websites that enforce that rule) to obviously naive (multiple purchases by a customer correlate to high loyalty) as well as highly useful to managing and running a business (there exists a single version of the truth for all data).  The value of useful simplifications can be seen in the creation of elegant architectures that enable business and IT to converse constructively about how to built systems the business can use.  They also reduce the complexity of the data systems; one size fits all.  The danger lies in the longer-term rigidity such simplifications can cause.

The data warehouse architecture of the 1980s, to which I was a major contributor, of course, was based largely on the above single-version-of-the-truth simplification.  There’s little doubt it has served us well.  But, big data and other trends are forcing us to look again at the underlying assumptions.  And find them lacking. IBM (and it’s not alone in this) has recognized that there exists different business use patterns of data which lead to different technology sweet spots.  The fundamental precept is not new, of course.  The division of computing into operational, informational and collaborative is closely related.  The new news is that the usage patterns are non-exclusive and overlapping; and they need to co-exist in any business of reasonable size and complexity.  I can identify four major business patterns: (1) mainstream daily processing, (2) core business monitoring and reporting, (3) real-time operational excellence and (4) data-informed planning and prediction.  And there are surely more.  This week, IBM announced three differently configured models: (1) PureData System for Transactions, (2) for Analytics and (3) Operational Analytics, each based on existing business use patterns and implementation expertise.  Details can be found here.  I imagine we will see further models in the future.

More Read

The State of Web3 Marketing for Data-Driven Businesses
The State of Web3 Marketing for Data-Driven Businesses
Applying Data Analytics to Customer Experience and Service on Social Media
Back to the Future of Business Intelligence
The N-gram and the Book “Uncharted: Big Data as a Lens on Human Culture”
10 Technology Trends that will Define Enterprise Architecture in the 2010s

All of this leads to a new architectural picture of the world of data–an integrated information platform, where we deliberately move form a layered paradigm to one of interconnected pillars of information, linked via integration, metadata and virtualization.  A more complete explanation can be found in my white paper, “The Big Data Zoo–Taming the Beasts:  The need for an integrated platform for enterprise information”.  As always, feedback is very welcome–questions, compliments and criticisms.

Share This Article
Facebook Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

Flat editorial illustration: The article connects downstream report failures, data-quality gaps, and conflicting metric definitio
Signs Your Data Architecture Needs an Outside Review
Big Data Exclusive
Data-Driven Pultrusion: Best GFRP Rebar Machines & ROI
Data-Driven Pultrusion: Best GFRP Rebar Machines & ROI
Big Data Exclusive
Flat editorial illustration: The article contrasts tools for gathering fresh interviews with tools for analyzing existing qualita
10 AI Market Research Tools That Turn Qualitative Data into Insights
Artificial Intelligence Big Data Exclusive
Flat editorial illustration: The article compares proxy MCP servers by what an AI agent can do through them: manage proxy account
Proxy MCP Servers: 7 Options and What Each Lets an AI Agent Do
Artificial Intelligence Big Data Data Collection

Stay Connected

1.2KFollowersLike
33.7KFollowersFollow
222FollowersPin

You Might also Like

Big Intelligence: Measuring the ROI of Your Hadoop Big Data Project
Big Data

Big Intelligence: Measuring the ROI of Your Hadoop Big Data Project

8 Min Read
GPS Data Is Incredibly Useful for Geolocation Optimization & Automotive Design
Big Data

GPS Data Is Incredibly Useful for Geolocation Optimization & Automotive Design

7 Min Read
Social Media Analytics – Understanding Your Social Customer and Context
Business IntelligenceData Mining

Social Media Analytics – Understanding Your Social Customer and Context

5 Min Read
Why Large Enterprises and EDW Owners Suddenly Care About Big Data
AnalyticsBest PracticesData WarehousingStatisticsText Analytics

Why Large Enterprises and EDW Owners Suddenly Care About Big Data

5 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

AI chatbots
AI Chatbots Can Help Retailers Convert Live Broadcast Viewers into Sales!
Chatbots
ai chatbot
How AI Website Chatbots Improve Customer Support and Lead Generation
Chatbots Exclusive

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-26 SmartData Collective. All Rights Reserved.
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?