The Future of Business Intelligence

Keynote address to the Next Generation Data Warehousing Conference, IDG Korea , Seoul, South Korea, February 16th 2011

I presented my thoughts to a couple of hundred Senior Managers from a wide range of Korean organisations. Crystal ball gazing is something I usually leave to vendors and consultants so I focussed on the following aspects that I at least have some practical experience of:

• Key trends to the end of the decade
• Best practices
• Organising the future intelligence capability.

The following key trends struck me as important factors influencing the future of our work in analytics:

• An Unprecedented Explosion In Data. “Between now and 2020, the amount of digital information created and replicated in the world will grow to an almost inconceivable 35 trillion gigabytes as all major forms of media”
  IDC iView, “The Digital Universe Decade – Are You Ready?” May 2010
• The Dominance of Mobile Multimedia Content. “Global mobile data traffic will increase 26-fold between 2010 and 2015 … Mobile network connection speeds will increase 10-fold by 2015 … Two-thirds of the world’s mobile data traffic will be video by 2015.”
  Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update, 2010–2015, February 2011
• The End Of English. Chinese-language content will dominate the internet by 2014
  Gartner 2009.

I’m not completely convinced about Chinese domination – but I agree that the Internet will become strongly multi-lingual as the rise of China and Brazil continues.  In any event, we all now live in a world of BIG data. Data that is growing annually by orders of magnitude and is moving to mobile/embedded devices that generate real-time multimedia data. On the other hand, IT skilled people – those with the skills most likely to handle data growth – are only predicted to grow 1.4 fold over the same period. So there is an emerging massive skills shortage coming and we have four choices:

• Do nothing. You wouldn’t be alone if this is your strategy.
• Focus your current analysts on embedding their skills in a broad range of your staff.
• Try to buy the skills by poaching staff from other organisations. This could get to be very expensive as you will likely to be in a bidding war with other organisations facing a similar skill shortage.
• Use software that enables your analysts to handle vastly more data than they can today.

Just to complicate things, the era of the English/Western Internet is being supplanted by Chinese – a language that is impervious to processing by most of the current and emerging intelligence capabilities out of the US.

Personally, I see as our best hope a strategy that focusses on embedding analytic skills into the general business community. This greatly enhances analytics usefulness in the short term, whether-or-not you believe in the above trends. It will also buy us the time to see smarter software emerge that does the ‘heavy lifting’ of BIG (Chinese?) data for analysts. The analogy I used in my address was transportation and the London underground network. The actual network looks like the following:

It took Harry Beck in 1931 to analyse the London underground network and realise that the actual physical location of the station was less important than the information about how you can get to your destination.

This is exactly what analytics needs to achieve. I like to call this actionable insight.

If you have any doubts about the explosion in data we can expect, take a look a recent years. The Washington Post has created a created a graphic that shows how data grew from 1996 and 2007:

After looking at the impact of these trends, I went on to discuss templates for both the analytic skills we need and how to best to organise your teams. I have posted about these topics previously here.

After the presentation there were questions from the floor and it was interesting to see that the questions could have come from any country. The problems and challenges discussed were exactly the same as I hear in other countries. It’s nice to know the we are all in the same boat.