From BI to Enterprise IT Integration

Autumn has arrived here in Cape Town and with it a slight chill in the air.  In a couple of weeks, I head up to Europe where spring is warming the land.  It’s a time of transitions.  And so it is too for the IT industry and BI in particular.

Autumn has arrived here in Cape Town and with it a slight chill in the air.  In a couple of weeks, I head up to Europe where spring is warming the land.  It’s a time of transitions.  And so it is too for the IT industry and BI in particular.

I’m presenting three seminars on the transformation of BI into Enterprise IT Integration in Europe: a half day each in Copenhagen and Helsinki (4 and 5 April) and a full two-day deep dive in Rome (11-12 April).  I started teaching these classes in early 2010, and as I review and update the contents, I’m struck by two impressions.  First, a lot has changed in the BI / Data Warehousing market in the past year.  Second, the new architecture, BI², that I’ve been developing since 2009 continues to closely map the emerging technology. That’s hugely gratifying!

So, what are the big changes?  One is the exploding interest in “Big Data“.  Now, as I’ve discussed elsewhere, the name is a total misnomer–the stuff is neither necessarily big nor data in the strict meaning of the words–but, the interesting thing is how the concepts and technologies around this area have energized thinking about BI.  Pre-defined reports have become even less interesting in the face of innovative data mining and predictive analytics in the big data environment.  And the information we can use to gain analytic value has expanded in variety and volumes beyond our wildest dreams.

Another important change has been the acquisitions of innovative start-ups, particularly in the analytic appliances field by both BI and other-than-BI giants–Netezza by IBM, Vertica by HP and Sybase by SAP to name a few. The acquisitions highlight the importance of these emerging technologies in a sea change in the way BI will be and increasingly is being delivered.

The third change is at the hardware level.  While much more is going on, two aspects are important to BI: the increasing number of cores in commodity processors and the move towards silicon-based data storage, whether on solid state disks or in memory.  Both of these trends suggest that internal database design approaches established since the 1970s are due for a big shake-up.

None of these trends is particularly new.  But their speed-up and convergence in the past year or so are rapidly pushing the BI environment to a tipping point–the architecture we’ve successfully used for over twenty years is rapidly being made irrelevant!  That architecture, dating from a 1988 paper that I co-authored, was based on business needs and technological solutions that were appropriate for their time.  Today’s bus-tech ecosystem is so different that I have been arguing for a number of years that we need to look beyond our current silos of operational systems, BI and office/collaborative systems to a new architecture that crosses all of IT.  This architecture, which I call Business Integrated Insight, proposes that we treat the entire information asset of the business as a single, logical entity–the Business Information Resource (BIR).

Such a change of focus has widespread implications for the entire IT support infrastructure.  The challenges are big, but the technology as it is emerging is enabling this change.  My European seminars describe the new architecture in depth, but, more importantly, link what needs to be done to current technologies and provides practical guidance on how to begin the transformation.