People, Process & Politics: Stop the (Integration) Madness

In previous post “People, Process and Politics – We All Hate Data Silos, So Why do They Happen?” we discussed why we have set ourselves up to continually repeat history and build data silos. Like an episode of The Twilight Zone we seem to be hopelessly locked into an endless loop.

Enterprises have fragmented their integration efforts across groups, applications and technologies. The classic split has been between the groups developing data warehouse and business intelligence applications versus the ERP (enterprise resource application) applications. Typically each group uses different technologies: ETL (extract, transform and load) for DW and applications using either data virtualization (formerly EAI Enterprise Application Integration) or EII (Enterprise Information Integration) used. Add in SOA (service oriented architecture) initiatives for application integration and an enterprise integration efforts are spread across multiple projects with different sponsors, funding, resources and objectives that often overlap.

If that is not a big enough problem, our industry has assaulted enterprises with a barrage of “emerging” technologies that are supposed to solve various integration . …

If that is not a big enough problem, our industry has assaulted enterprises with a barrage of “emerging” technologies that are supposed to solve various integration problems. These solutions include Corporate Performance Management (CPM), Master Data Management (MDM), Customer Data Integration (CDI), Product Information Management (PIM), Enterprise Information Management (EIM) and many others. Each of these solves problems in specific domains but at their core they involve integration. Vendors selling solutions bundle integration technologies (the same as described above) with their solutions. The result is integration silos that create new data silos.

Admit It, You’ve Got a Problem

You must admit you have a problem before you can solve it. Many enterprises are blind to their integration silos. All they see is their investments in ERP, DW, BI, CPM, MDM, CDI, SOA and PIM applications and the resulting databases with terabytes of data stored in them. Smug with the knowledge that they have all the data that the business needs, they’re not even aware of the data silos surrounding them created by their integration silos.

At some point, they start noticing inconsistent data, which is a symptom of an integration silo problem. However, because they don’t understand the cause, they focus on relieving the symptoms with quick-hit solutions. For example, they may try to consolidate business intelligent (BI) tools. While this is a worthy goal unto itself, using a single BI tool will do nothing about the fact that the underlying data is coming from many disparate systems ERP, CRM, SCM, data warehouses, data marts) and is inconsistent. The business is not getting different numbers because it is using different BI tools, but rather because each tool is associated with a different database where the data had been transformed differently than the other databases. The BI tool used is the tip of the iceberg; the data integration issue is what is below the waterline.

Once you recognize an integration silo problem, you can think out-of-the-box on how to define, approach and attack it.

Next post on “People, Process & Politics” we will begin to discuss how to stop the integration madness.