Custom code and data integration thoughts

Julie Hunt (@juliebhunt) wrote a piece recently on Custom Code or Data Integration software tools in which she made some great points about the excessive use of custom code when integrating data. Two things I would like to add:

Julie Hunt (@juliebhunt) wrote a piece recently on Custom Code or Data Integration software tools in which she made some great points about the excessive use of custom code when integrating data. Two things I would like to add:

1. One of the drivers for custom code is that the integration of data requires a complex business decision. For instance, I remember a financial services company that was doing data integration for its P&L and had to make a complex decision for each subsidiary as to exactly how the data from that subsidiary impacted the overall P&L. If this is driving custom coding in your data integration project then consider externalizing these decisions using business rules. For instance, my friends at IDIOM (a client of mine) have partnered with the folks at Talend to bring decisioning to the data integration process.
2. One way to effectively focus data integration efforts, and to more fully engage business people, is to be absolutely clear which business decision you are working to improve. Not only does this make the project more business focused (it becomes about improving the decision not about technical integration), it also makes it clear which data is actually required. I blogged about the importance of keeping the decision in mind when doing data integration before.