Cookies help us display personalized product recommendations and ensure you have great shopping experience.

By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    data driven insights
    How Data-Driven Insights Are Addressing Gaps in Patient Communication and Equity
    8 Min Read
    pexels pavel danilyuk 8112119
    Data Analytics Is Revolutionizing Medical Credentialing
    8 Min Read
    data and seo
    Maximize SEO Success with Powerful Data Analytics Insights
    8 Min Read
    data analytics for trademark registration
    Optimizing Trademark Registration with Data Analytics
    6 Min Read
    data analytics for finding zip codes
    Unlocking Zip Code Insights with Data Analytics
    6 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: What Does Taming Big Data Really Cost?
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Analytics > What Does Taming Big Data Really Cost?
AnalyticsBig Data

What Does Taming Big Data Really Cost?

BillFranks
BillFranks
6 Min Read
Image
SHARE

ImageI’ve written in the past about the hype that surrounds big data. Perhaps one of the most ridiculous urban legends out there today is that working with big data is cheap. This myth is largely driven by the fact that some big data tools, such as Hadoop, don’t come with a license fee.

ImageI’ve written in the past about the hype that surrounds big data. Perhaps one of the most ridiculous urban legends out there today is that working with big data is cheap. This myth is largely driven by the fact that some big data tools, such as Hadoop, don’t come with a license fee.

Before I get started, let me be clear on one thing. Open source products such as Hadoop have many uses and can play important roles in the development of analytic processes. Plus, us stats geeks love to download useful software for free just like anyone else. Who doesn’t want free stuff? The key is to look at what it will cost in total to actually use that software in the context in which you plan to use it. If you aren’t careful to take into account the entire spectrum of costs for your analytic platform, you might just find yourself going way over your planned budget.

Many Price Is Right winners have learned the hard way that their “free” RV came with a large tax bill and heavy ongoing maintenance and operational costs. If you can’t handle those costs, then winning an RV isn’t as exciting as it seems. Similarly, getting a free puppy leads to a lot of ongoing work, as well as costs for food, vets, and myriad other items. It isn’t that winning an RV, or getting a puppy, or deciding to leverage Hadoop is a bad thing. It is simply a matter of being sure you understand what you’re committing to.

More Read

big data
5 Ways Big Data Is Being Used To Understand COVID-19
How to Pivot Your Data
Social Media Analytics: How our approach blends the best analytics technology
Big Data is Not Just Hadoop
How BI and Data Analytics Professionals Used Twitter in February

Look Beyond License Fees

Regardless of what you pay for your software, license fees are but the beginning of the overall cost of operation for an analytic platform. Other costs include:

  • The hardware that the software will be installed on
  • The space taken and power used by the hardware
  • Configuring & implementing security, resource prioritization, and other operational features
  • Acquiring, loading, and making data ready for use
  • Developing analytic processes on your platform
  • The latency between requests and results being delivered
  • Maintaining the platform
  • Training staff to use and configure the platform
  • Consultants needed for any part of implementation

Certainly there are other costs, but the point should be clear. You can’t simply look at any one factor when determining your costs and deciding which direction to go. A simple comparison of license fees or cost per server won’t lead you to the right decision. In addition, it is also important to look at total cost over the expected life of the platform. Saving on license fees can lower first year costs, but if ongoing costs are more expensive, that can be more than made up for over time.

I believe that the most often underestimated line items relate to the man hours required from employees or consultants to actually stand up, configure, utilize, and maintain an environment. It is absolutely critical to account for these costs. This is one area, for example, that Hadoop can run up your bill. It is a newer, maturing technology that few people are yet familiar with. This leads to a steep learning curve. Again, this isn’t to say the learning curve can’t be worth it. I am simply saying that you have to recognize and account for it.

One of the best discussions I have seen on this topic is a recent paper from Richard Winter at Winter Corp. I encourage you to download and read his report. In it, he develops a framework and provides some examples that lead to very different conclusions on where to invest based on the profile of the data being targeted and the analytics required. He shows how in some cases, Hadoop is a solid fit and in others it doesn’t fit well at all. That’s also true with other technologies. As always, it comes down to what you need to do.

As Teradata states with its Unified Data Architecture, there is a place for Hadoop and other open source products. There is also a place for commercial technologies such as Teradata. Each has a unique total cost profile based upon the type and volume of processing required. By focusing on your total costs, you’ll be better able to allocate your resources over time and you’ll ensure that the right tools and technologies are being used in the right ways.

If you’re reading this, your action item is to make sure your organization is considering all the costs it will incur over time in order to make a solid, appropriate decision as to what tools and technologies serve what roles in your environment. Otherwise, you’ll make it harder for your organization to tame big data.

Originally published by the International Institute for Analytics

Share This Article
Facebook Pinterest LinkedIn
Share
ByBillFranks
Follow:
Bill Franks is Chief Analytics Officer for The International Institute For Analytics (IIA). Franks is also the author of Taming The Big Data Tidal Wave and The Analytics Revolution. His work has spanned clients in a variety of industries for companies ranging in size from Fortune 100 companies to small non-profit organizations. You can learn more at http://www.bill-franks.com.

Follow us on Facebook

Latest News

accountant using ai
AI Improves Integrity in Corporate Accounting
Exclusive
ai and law enforcement
Forensic AI Technology is Doing Wonders for Law Enforcement
Artificial Intelligence Exclusive
langgraph and genai
LangGraph Orchestrator Agents: Streamlining AI Workflow Automation
Artificial Intelligence Exclusive
ai fitness app
Will AI Replace Personal Trainers? A Data-Driven Look at the Future of Fitness Careers
Artificial Intelligence Big Data Exclusive

Stay Connected

1.2kFollowersLike
33.7kFollowersFollow
222FollowersPin

You Might also Like

predictive analytics used by amazon
AnalyticsPredictive Analytics

Amazon Wants to Use Predictive Analytics to Offer Anticipatory Shipping

6 Min Read

How to Do Data Analytic: A 101 Crash Course for CEOs

10 Min Read

ViDEO: Change Your Business Results by Using Social Analysis

2 Min Read

The importance of feasibility studies in business intelligence

17 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

AI chatbots
AI Chatbots Can Help Retailers Convert Live Broadcast Viewers into Sales!
Chatbots
ai in ecommerce
Artificial Intelligence for eCommerce: A Closer Look
Artificial Intelligence

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-25 SmartData Collective. All Rights Reserved.
Go to mobile version
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?