By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData CollectiveSmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    data-driven white label SEO
    Does Data Mining Really Help with White Label SEO?
    7 Min Read
    marketing analytics for hardware vendors
    IT Hardware Startups Turn to Data Analytics for Market Research
    9 Min Read
    big data and digital signage
    The Power of Big Data and Analytics in Digital Signage
    5 Min Read
    data analytics investing
    Data Analytics Boosts ROI of Investment Trusts
    9 Min Read
    football data collection and analytics
    Unleashing Victory: How Data Collection Is Revolutionizing Football Performance Analysis!
    4 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: Build vs. Buy? Things to Consider Before Building an In-House Analytics System
Share
Notification Show More
Aa
SmartData CollectiveSmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Analytics > Build vs. Buy? Things to Consider Before Building an In-House Analytics System
Analytics

Build vs. Buy? Things to Consider Before Building an In-House Analytics System

Puneet Pandit
Last updated: 2013/09/17 at 8:00 AM
Puneet Pandit
4 Min Read
Image
SHARE

ImageMany analytics projects involving log files focus on operational event data. IT use cases around such projects typically focus on locating errors, warnings and critical event information within mountains of data. However, software applications and technology devices produce much more machine data than just log files, which can be incredibly complex.

ImageMany analytics projects involving log files focus on operational event data. IT use cases around such projects typically focus on locating errors, warnings and critical event information within mountains of data. However, software applications and technology devices produce much more machine data than just log files, which can be incredibly complex.

Organizations need to search and mine information contained in machine data and have two choices to execute this challenge: 1) obtain a purpose-built analytics system or 2) build one in-house. While the latter option may sound appealing at first, here are some things to consider before building an in-house machine analytics system:

1. Different users have different requirements

More Read

data-driven white label SEO

Does Data Mining Really Help with White Label SEO?

IT Hardware Startups Turn to Data Analytics for Market Research
The Power of Big Data and Analytics in Digital Signage
Data Analytics Boosts ROI of Investment Trusts
Unleashing Victory: How Data Collection Is Revolutionizing Football Performance Analysis!

A machine data analytics solution must satisfy the requirements of a wide range of internal consumers. A support engineer working a case needs to be able to see patterns of events and statistics over time grouped by specifics system components. An account manager, professional services engineer or sales rep has very different data analysis needs that involve being able to quickly spot reliability, performance and other issues involving an account, while a marketing or program manager needs spotting feature adoption and trends in product demand. This differentiation between jobs and teams highlights the complexity of an analytics system through machine data.

 2. Every product has a complex and unique representation for machine data

For a set of data to be useful, it needs to communicate detailed information about the configuration, events, and statistics of each product’s unique architecture. Applications, appliances, software companies always log events, counters and information about internal component states, down to the level of vendor-specific abstractions. Much of this data will be specific and will not conform to the common information model for interoperability or end use.

3. A useful data archive will be “Big Data

To be of maximal benefit to all consumers, a good set of machine data need to contain “everything.” Trending and field analysis require detailed parsing for all systems continuously, not just when problems are detected. A successful product may have thousands or even millions of devices and systems reporting data back regularly. The volume of data received and retained in such a case is likely to be in the range of hundreds of terabytes or petabytes over the course of a year.

4. Data formatting and semantics will change quickly and without notice

Machine data analytics require quick adaptation, edge-case coverage and continuous business leverage. A machine data analytics solution cannot expect schematized for specially formatted data, it needs to adapt quickly to changes in format of information from machine logs while maintaining semantic continuity with existing tools. 

In-house solution: The bottom line

While companies can choose to build such an analytical solution in house, it’s not worth the time and effort to do so. An in-house machine data analytics solution is a complex high-performance big data project associated BI tools that requires a variety of committed resources for an extended period of time. It’s inherently time-consuming and risky if not planned properly with appropriate resources needed to design and implement, but also to maintain and manage its life cycle on a continued basis.

Puneet Pandit September 17, 2013
Share This Article
Facebook Twitter Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

iot and cloud technology
IoT And Cloud Integration is the Future!
Internet of Things
ai in marketing
4 Ways AI Can Improve Your Marketing Strategy
Artificial Intelligence
data security unveiled
Data Security Unveiled: Protecting Your Information in a Connected World
Security
it management for data-driven businesses
7 Major IT Infrastructure Challenges for Data-Driven Companies
IT

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

data-driven white label SEO
Analytics

Does Data Mining Really Help with White Label SEO?

7 Min Read
marketing analytics for hardware vendors
Analytics

IT Hardware Startups Turn to Data Analytics for Market Research

9 Min Read
big data and digital signage
Analytics

The Power of Big Data and Analytics in Digital Signage

5 Min Read
data analytics investing
Analytics

Data Analytics Boosts ROI of Investment Trusts

9 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

giveaway chatbots
How To Get An Award Winning Giveaway Bot
Big Data Chatbots Exclusive
data-driven web design
5 Great Tips for Using Data Analytics for Website UX
Big Data

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?