Sybase: Big Data Crisis is a Big Lie

Sybase (“an SAP Company”) recently published an analytics guide called “Intelligence for Everyone”.

The first section called “The Big Lie about Big Data” includes the following:

Why “big lie?” Because, as the article points with lots of historical examples, there’s always been a “crisis” in data storage, but as data volumes have risen technology has always evolved to deal with it, and that’s as true today as it has been in the past:

• 1956 Transaction data volumes overloaded current memory systems, leading to IBM’s creation of the first hard drive (costing $10,000 per Mb)
• 1970 Alvin Toffler’s book Future Shock popularizes the phrase “information overload”
• 1986 Technology critic Theodore Roszak:  “An excess of information may actually crowd out ideas, leaving the mind… distracted by sterile, disconnected facts, lost among shapeless heaps of data.”
• 1990 An IEEE conference features a session “Crisis in Mass Storage.”
• 1995 A Montreal data mining conference talks about the “data firehose phenomenon” swamping users

Interestingly enough, the guide manages to talk a lot about Big Data without mentioning what many people associate the term with: open source technologies such as Hadoop/Map Reduce. This is all the stranger because the latest version of Sybase IQ 15.4 includes a native MapReduce API and Hadoop integration, getting closer to the ideal architectures of the future that take the best of both worlds.

The guide also slides over the advantages of in-memory storage such as SAP HANA, noting just that “64-bit systems with their larger RAM space make this technology more attractive, if more expensive”

For more on SAP HANA and Sybase IQ, here’s analyst group Ovum’s take: “SAP’s HANA and Sybase IQ are separate but complementary“

The guide concludes that:

“Big Data is not to be feared. It’s to be exploited. The analytics industry today has no excuses when it comes to working with Big Data. It has no excuses when it comes to scaling their analytics data warehouse to include thousands of users. It has no excuses when it comes to applying analytics to variable data types from every imaginable source, including, for example, the vast unstructured information from social media sites.”