Big Data: A Kick in the Business Intelligence Expert’s Habits

For some time the word Big Data appeared and is widely used by journalists, analysts, consultants and some software or hardware vendors interested in the world of BI. No definition has emerged and speeches about Big Data mix many things: overall volume of data to be processed, the volume of the elementary data (web log, text, sensors networks, photo, video), data types (structured, unstructured, multi-structured) and analytical ambitions (going beyond BI), etc.

For some time the word Big Data appeared and is widely used by journalists, analysts, consultants and some software or hardware vendors interested in the world of BI. No definition has emerged and speeches about Big Data mix many things: overall volume of data to be processed, the volume of the elementary data (web log, text, sensors networks, photo, video), data types (structured, unstructured, multi-structured) and analytical ambitions (going beyond BI), etc.

It is unclear whether the word Big Data is adapted and will continue, but it is certain that simultaneously demand for decision support is growing and new solutions offers appear (what about the chicken and the egg). “New” requests and “new” technology solutions push to process more data both in terms of volume and variety. Volume of data generated daily in information systems is growing exponentially and therefore volume explodes also for decision support systems. Ten years ago Teradata animated a club composed of clients who had more than a Terabyte in their decision system; today we have a club for companies which have more than one Petabyte.

What seems to me most important in all these new approaches is not necessarily the volume, but the desire to develop advanced analytical approaches in dealing with all kinds of raw data, which require a lot of work to draw business information. No longer you can take only some invoice lines and do some basic operations to generate a more or less aggregated data which has business value. For example you could want to identify a customer / prospect surfing the web, define if he has a positive or negative image of your brand or product, or identify networks of friends or defrauders who could create business concerns, or implement engines recommendations that are based on customers’ web browsing and profile, etc.…

 In order to have functionalities mentioned above, you have to get a set of tools that extract raw data (e.g. web logs, texts, social networks), to gain knowledge (profile, segment, affinity, churn), to predict (attrition, propensity, virality) and act (recommendations for tenders, pricing). You have to be able to mix data from a classic intelligence information system, with data extracted and structured via MapReduce (Hadoop, Aster Data) programs. Beyond data integration you have to work with data mining tools for advanced analysis, develop models and use them to enhance business processes, for example on websites, call centers, or your various channel interactions with the customer.

To go further on Teradata Aster specialized patented solution offering SQL-MapReduce ™, you can usefully consult the following link, and discover why companies like LinkedIn, Gilt Groupe, and Barnes & Noble have tapped the data deluge for competitive advantage with a Teradata Aster solution: http://www.asterdata.com/product/index.php