Cookies help us display personalized product recommendations and ensure you have great shopping experience.

By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    data analytics and truck accident claims
    How Data Analytics Reduces Truck Accidents and Speeds Up Claims
    7 Min Read
    predictive analytics for interior designers
    Interior Designers Boost Profits with Predictive Analytics
    8 Min Read
    image fx (67)
    Improving LinkedIn Ad Strategies with Data Analytics
    9 Min Read
    big data and remote work
    Data Helps Speech-Language Pathologists Deliver Better Results
    6 Min Read
    data driven insights
    How Data-Driven Insights Are Addressing Gaps in Patient Communication and Equity
    8 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: What Does Taming Big Data Really Cost?
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Analytics > What Does Taming Big Data Really Cost?
AnalyticsBig Data

What Does Taming Big Data Really Cost?

BillFranks
BillFranks
6 Min Read
Image
SHARE

ImageI’ve written in the past about the hype that surrounds big data. Perhaps one of the most ridiculous urban legends out there today is that working with big data is cheap. This myth is largely driven by the fact that some big data tools, such as Hadoop, don’t come with a license fee.

ImageI’ve written in the past about the hype that surrounds big data. Perhaps one of the most ridiculous urban legends out there today is that working with big data is cheap. This myth is largely driven by the fact that some big data tools, such as Hadoop, don’t come with a license fee.

Before I get started, let me be clear on one thing. Open source products such as Hadoop have many uses and can play important roles in the development of analytic processes. Plus, us stats geeks love to download useful software for free just like anyone else. Who doesn’t want free stuff? The key is to look at what it will cost in total to actually use that software in the context in which you plan to use it. If you aren’t careful to take into account the entire spectrum of costs for your analytic platform, you might just find yourself going way over your planned budget.

Many Price Is Right winners have learned the hard way that their “free” RV came with a large tax bill and heavy ongoing maintenance and operational costs. If you can’t handle those costs, then winning an RV isn’t as exciting as it seems. Similarly, getting a free puppy leads to a lot of ongoing work, as well as costs for food, vets, and myriad other items. It isn’t that winning an RV, or getting a puppy, or deciding to leverage Hadoop is a bad thing. It is simply a matter of being sure you understand what you’re committing to.

More Read

IBM Global Chief Supply Chain Officer Study 2009 View the…
Using Data Annotations for Quality Control Purposes
HealthMiner, an application that analyzes patient data, was…
The ABCs of Agile BI
Enterprise Data Trends to Watch for in 2014

Look Beyond License Fees

Regardless of what you pay for your software, license fees are but the beginning of the overall cost of operation for an analytic platform. Other costs include:

  • The hardware that the software will be installed on
  • The space taken and power used by the hardware
  • Configuring & implementing security, resource prioritization, and other operational features
  • Acquiring, loading, and making data ready for use
  • Developing analytic processes on your platform
  • The latency between requests and results being delivered
  • Maintaining the platform
  • Training staff to use and configure the platform
  • Consultants needed for any part of implementation

Certainly there are other costs, but the point should be clear. You can’t simply look at any one factor when determining your costs and deciding which direction to go. A simple comparison of license fees or cost per server won’t lead you to the right decision. In addition, it is also important to look at total cost over the expected life of the platform. Saving on license fees can lower first year costs, but if ongoing costs are more expensive, that can be more than made up for over time.

I believe that the most often underestimated line items relate to the man hours required from employees or consultants to actually stand up, configure, utilize, and maintain an environment. It is absolutely critical to account for these costs. This is one area, for example, that Hadoop can run up your bill. It is a newer, maturing technology that few people are yet familiar with. This leads to a steep learning curve. Again, this isn’t to say the learning curve can’t be worth it. I am simply saying that you have to recognize and account for it.

One of the best discussions I have seen on this topic is a recent paper from Richard Winter at Winter Corp. I encourage you to download and read his report. In it, he develops a framework and provides some examples that lead to very different conclusions on where to invest based on the profile of the data being targeted and the analytics required. He shows how in some cases, Hadoop is a solid fit and in others it doesn’t fit well at all. That’s also true with other technologies. As always, it comes down to what you need to do.

As Teradata states with its Unified Data Architecture, there is a place for Hadoop and other open source products. There is also a place for commercial technologies such as Teradata. Each has a unique total cost profile based upon the type and volume of processing required. By focusing on your total costs, you’ll be better able to allocate your resources over time and you’ll ensure that the right tools and technologies are being used in the right ways.

If you’re reading this, your action item is to make sure your organization is considering all the costs it will incur over time in order to make a solid, appropriate decision as to what tools and technologies serve what roles in your environment. Otherwise, you’ll make it harder for your organization to tame big data.

Originally published by the International Institute for Analytics

Share This Article
Facebook Pinterest LinkedIn
Share
ByBillFranks
Follow:
Bill Franks is Chief Analytics Officer for The International Institute For Analytics (IIA). Franks is also the author of Taming The Big Data Tidal Wave and The Analytics Revolution. His work has spanned clients in a variety of industries for companies ranging in size from Fortune 100 companies to small non-profit organizations. You can learn more at http://www.bill-franks.com.

Follow us on Facebook

Latest News

data analytics and truck accident claims
How Data Analytics Reduces Truck Accidents and Speeds Up Claims
Analytics Big Data Exclusive
predictive analytics for interior designers
Interior Designers Boost Profits with Predictive Analytics
Analytics Exclusive Predictive Analytics
big data and cybercrime
Stopping Lateral Movement in a Data-Heavy, Edge-First World
Big Data Exclusive
AI and data mining
What the Rise of AI Web Scrapers Means for Data Teams
Artificial Intelligence Big Data Exclusive

Stay Connected

1.2kFollowersLike
33.7kFollowersFollow
222FollowersPin

You Might also Like

Disaster Recovery Solutions in Cloud Infrastructure

8 Min Read
stream processing
Big DataExclusivePrivacy

Big Data Sets New Standards In Stream Processing For Emerging Markets

7 Min Read
analytics for recovery point objectives
Predictive Analytics

Growing Importance Of Predictive Analytics For Recovery Point Objectives

7 Min Read

Take the Predictive Analytics World Survey

1 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

ai chatbot
The Art of Conversation: Enhancing Chatbots with Advanced AI Prompts
Chatbots
AI and chatbots
Chatbots and SEO: How Can Chatbots Improve Your SEO Ranking?
Artificial Intelligence Chatbots Exclusive

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-25 SmartData Collective. All Rights Reserved.
Go to mobile version
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?