Cookies help us display personalized product recommendations and ensure you have great shopping experience.

By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    sales and data analytics
    How Data Analytics Improves Lead Management and Sales Results
    9 Min Read
    data analytics and truck accident claims
    How Data Analytics Reduces Truck Accidents and Speeds Up Claims
    7 Min Read
    predictive analytics for interior designers
    Interior Designers Boost Profits with Predictive Analytics
    8 Min Read
    image fx (67)
    Improving LinkedIn Ad Strategies with Data Analytics
    9 Min Read
    big data and remote work
    Data Helps Speech-Language Pathologists Deliver Better Results
    6 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: Big Data’s Big Flip-Flop
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Analytics > Big Data’s Big Flip-Flop
AnalyticsBig Data

Big Data’s Big Flip-Flop

BillFranks
BillFranks
6 Min Read
SHARE

Analytics Matters with Bill Franks

Analytics Matters with Bill Franks

It wasn’t too long ago that many people espoused the decline, if not death, of the SQL language and relational database technology in general. As a level set, remember that relational technology stores data into rows and columns and that the way to access relational data is through Structured Query Language (SQL). For a couple of years, there was a full frontal assault on relational approaches from the Hadoop and non-relational crowds. The overhead of placing data into pre-defined rows and columns was deemed too great, compared to storing data within a non-relational environment.

In non-relational environments, users are free to use a wide range of programming languages to analyze data in any format. The data is typically simply stored in files with no assumed format or relationships. This approach does have its merits, but it also has its limitations.

More Read

Google Paper on Parallel EM Algorithm using MapReduce
Why Data-Driven Businesses Need Clean Sales Data
What is Cloud Computing
Predictive Analytics Drives Criminal Justice Reform with Recidivism Forecasting
Top 5 Tools for Building an Interactive Analytics App

In case you hadn’t noticed, a huge flip-flop has occurred. Many of the same people and organizations that were recently dismissing the entire concept of relational environments and SQL are now racing to … wait for it … add SQL-style interfaces on top of non-relational platforms like Hadoop! Let’s first take a look at how the flip-flop came about and then discuss why it is a good thing.

One big and mistaken assumption in the case against relational technologies is that relational technologies are not flexible and can’t handle unexpected questions or poorly formatted data. Therefore, a non-relational platform is required to be nimble. It is important to distinguish between an inherent shortcoming of a relational system and a shortcoming in how that system is implemented. That distinction is critical to understanding the flip-flop.

It is true that many organizations, particularly the large ones, not only had a large number of relational systems in place, but also locked the systems down very tightly. It was in fact difficult for users to ask new questions or to gain access to enough computing resources. However, this was due to the policies laid on top of relational technology as opposed to the technology itself. It is entirely possible to load and query data in a relational environment that isn’t in 3rd normal form, that hasn’t been formally modeled, and that isn’t yet clean. I spent years doing this.

The concept of an analytic sandbox or discovery environment centers on freeing users from traditional IT-imposed access limits and allowing them to explore and experiment with data in a relational environment. Granted, not all types of data can be handled in a relational system, but most common business data sources can be.

Like any solution, relational approaches are very good for many problems and are not as good for others. The same can be said about non-relational environments. Analytic professionals like me have always used a mix of environments because it isn’t about one approach being better or worse, but about which fits a given problem best. To me, SQL is the new kid on the block because when I started out, SQL did not exist! Over time, I added SQL processing into the mix where it made sense. It ended up making sense a huge proportion of the time, but not all of the time.

Recently, some organizations have tried to do too much with non-relational platforms. In many cases, this has led to inefficient processes that take more time to create, manage, and process than standard SQL approaches. Luckily, most of those who were looking to put up SQL’s tombstone have come around to their error.

It is terrific for the industry that the flip-flop around relational technologies has occurred. Having a mix of capabilities is a good thing and it isn’t a zero-sum game where only one approach can win. Facebook realized that trying to implement SQL-style processing outside of an environment built for it was wasting time and money to reinvent something that already existed and worked just fine. As a result, Facebook added a large relational environment into its mix because certain types of processing just work better that way.

I’ll be participating in a virtual event March 27 called Data Discovery In Action. Feel free to register here at no cost. The focus of the event will be on how to combine various processing paradigms and analytic techniques to maximize the ability of your organization to discover and deploy new high impact analytics. There will be discussion of both relational and non-relational approaches, which is how it should be!

Many of us who have spent years developing advanced analytic processes were surprised to see relational technologies and SQL getting beat up so badly. It never made sense to kill SQL and I’ll forgive those who were misguided in their attempts to do so. After all, it can’t help but sting a little to have to pull an about face and execute a flip-flop like politicians are known to do. But, sometimes executing a flip-flop is the right thing to do.

Share This Article
Facebook Pinterest LinkedIn
Share
ByBillFranks
Follow:
Bill Franks is Chief Analytics Officer for The International Institute For Analytics (IIA). Franks is also the author of Taming The Big Data Tidal Wave and The Analytics Revolution. His work has spanned clients in a variety of industries for companies ranging in size from Fortune 100 companies to small non-profit organizations. You can learn more at http://www.bill-franks.com.

Follow us on Facebook

Latest News

sales and data analytics
How Data Analytics Improves Lead Management and Sales Results
Analytics Big Data Exclusive
ai in marketing
How AI and Smart Platforms Improve Email Marketing
Artificial Intelligence Exclusive Marketing
AI Document Verification for Legal Firms: Importance & Top Tools
AI Document Verification for Legal Firms: Importance & Top Tools
Artificial Intelligence Exclusive
AI supply chain
AI Tools Are Strengthening Global Supply Chains
Artificial Intelligence Exclusive

Stay Connected

1.2kFollowersLike
33.7kFollowersFollow
222FollowersPin

You Might also Like

Image
ExclusivePredictive Analytics

Inside a Consumer’s Mind with Text Analytics

5 Min Read
big data companies
Big Data

8 Amazing Big Data Companies You Should Know, But Probably Don’t

7 Min Read

Walmart Makes Big Data Part of Its DNA

6 Min Read
data encryption for data security
Analytics

Understanding the Nature and Evolution of Security Analytics

8 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

giveaway chatbots
How To Get An Award Winning Giveaway Bot
Big Data Chatbots Exclusive
data-driven web design
5 Great Tips for Using Data Analytics for Website UX
Big Data

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-25 SmartData Collective. All Rights Reserved.
Go to mobile version
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?