We use cookies, including third-party cookies from Google to serve personalized ads through AdSense, to operate this site and understand how it is used. By continuing to browse, you accept this use. See our Privacy Policy and Terms of Use for details, including how to opt out of personalized advertising.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    Flat editorial illustration: The article is about joining public web signals with internal business metrics so BI teams can inter
    Alternative Data in BI: Bringing Public Web Data Into Your Dashboards
    23 Min Read
    Flat editorial illustration: The article is about automating the collection, preparation, and delivery of marketing data across p
    From Data to Dashboards: 5 Marketing Reporting Automation Software Picks
    12 Min Read
    Flat editorial illustration: The article contrasts passive follower totals with a behavioral path from attention and trust to cli
    Creator Analytics Beyond Follower Count: Seven Metrics That Measure Audience Behavior
    11 Min Read
    Flat editorial illustration: The article connects AI and marketing analytics with managerial judgment grounded in consumer behavi
    How an Online MBA in Marketing Connects AI and Data Analytics
    11 Min Read
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers -- AI-generated illustration
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers
    7 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: Big Data
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Business Intelligence > Big Data
Business Intelligence

Big Data

Steve Bennett
Steve Bennett
6 Min Read
Big Data
Illustration generated with FLUX.1 [schnell] via Cloudflare Workers AI.
SHARE

Several month’s ago a short video appeared on YouTube with an interview of LinkedIn’s Chief Scientist DJ Patil. In it he discusses how ‘Big Data‘ impacts the practise of analytics. I’ve only just got around to posting about it but I am doing so now because he has some insights that I agree with and would like to share as they are still relevant. 

Big data is today most often associated with the internet superstars like Google, eBay and Amazon. There are 3 other areas with lower profiles where big data is important: intelligence (spooks, the military, etc.), scientific and academic research, and the financial markets.

Big data’s future is much bigger than this because more and more areas of human activity are going to be faced with vast data sets. When you hear people talking about the growth of knowledge and statements like ‘if this data were printed then the stack would grow faster than NASA’s fastest rocket‘, you have to remember that there is a good chance that each page of new data is adding to someone’s analytic data set.

    I’m not quoting the guy verbatim but here’s what I heard and my takeouts to his comments:

    • Open source ‘big data ready’ technologies like Hadoop (see my earlier blog or here) have come into their own now. Look to people with these skills over those only with SQL if you are facing big data challenges.
    • We have reached a tipping point in the use of open source for commercial solutions to big data problems.
    • If you want good analysts then the best place is to look is in occupations where people will already have the practical skills in manipulating big data sets: scientific fields like meteorology, oceanography and the like. I agree but this is not the only place as in my experience I also need analysts that relate well to business decision makers – i.e. those people that make commercial decisions based on the analytics. This is perhaps less important in pure tech plays like LinkedIn.
    • Open source will transform the practise of analytics in the next 3 – 5 years. I think it will take longer than this to really impact the more traditional industries. I’m not happy about this but I am realistic about the difficulty in convincing business leaders that open source is a superior solution to proprietary ones. The money behind the big vendors will keep them going for a number of years yet.

    One potential qualifier to DJ Patil’s perspective is that although he has a very impressive big data background as a mathematician, US Department of Defence analyst (‘Threat Anticipation’), and former eBay Director of Strategy and Analytics, his current employer is LinkedIn.

    The core of LinkedIn’s big data is structured and fairly static: profiles of people. So I’m not sure how similar their big data challenges would be to, say, those faced with processing, understanding and predicting large streams of real time data from financial markets or very large sensor arrays. On the other hand, the growth of LinkedIn communities and their related activities must generate large amounts of semi-structured data.

    I also have no idea what LinkedIn’s own analytic goals are beyond what DJ mentions on his own profile where he says his analytics drives product features like:

    • “People You May Know”
    • “Who Viewed My Profile”
    • “Groups You Might Like”

    Maybe somebody reading this blog knows more?

    The video is on YouTube and I embed it here for convenience:

    Or you can download ‘DJ Patil on How Big Data Impacts Analytics’ directly from this blog.

    TAGGED:data qualityhadoopopen source
    Share This Article
    Facebook Pinterest LinkedIn
    Share

    Follow us on Facebook

    Latest News

    b2b payment data tells supplier about its customers
    What B2B Payment Data Tells a Supplier About Its Customers
    Business Intelligence
    Flat editorial illustration: The article connects downstream report failures, data-quality gaps, and conflicting metric definitio
    Signs Your Data Architecture Needs an Outside Review
    Big Data Exclusive
    Data-Driven Pultrusion: Best GFRP Rebar Machines & ROI
    Data-Driven Pultrusion: Best GFRP Rebar Machines & ROI
    Big Data Exclusive
    Flat editorial illustration: The article contrasts tools for gathering fresh interviews with tools for analyzing existing qualita
    10 AI Market Research Tools That Turn Qualitative Data into Insights
    Artificial Intelligence Big Data Exclusive

    Stay Connected

    1.2KFollowersLike
    33.7KFollowersFollow
    222FollowersPin

    You Might also Like

    Information Theory Approach to Data Quality and MDM
    Uncategorized

    Information Theory Approach to Data Quality and MDM

    15 Min Read
    Data Visualization: Why (1 of 2)
    Data Visualization

    Data Visualization: Why (1 of 2)

    8 Min Read
    Flat editorial illustration: A data integration ecosystem showing the six interconnected components (sources, big data storage, d
    AnalyticsBig DataData ManagementExclusiveIT

    Data Integration Ecosystem for Big Data and Analytics

    10 Min Read
    TDWI World Conference Chicago 2009
    Business IntelligenceData Warehousing

    TDWI World Conference Chicago 2009

    14 Min Read

    SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

    AI chatbots
    AI Chatbots Can Help Retailers Convert Live Broadcast Viewers into Sales!
    Chatbots
    The Art of Conversation: Enhancing Chatbots with Advanced AI Prompts
    The Art of Conversation: Enhancing Chatbots with Advanced AI Prompts
    Chatbots

    Quick Link

    • About
    • Contact
    • Privacy
    Follow US
    © 2008-26 SmartData Collective. All Rights Reserved.
    Welcome Back!

    Sign in to your account

    Username or Email Address
    Password

    Lost your password?