We use cookies, including third-party cookies from Google to serve personalized ads through AdSense, to operate this site and understand how it is used. By continuing to browse, you accept this use. See our Privacy Policy and Terms of Use for details, including how to opt out of personalized advertising.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    Flat editorial illustration: The article is about joining public web signals with internal business metrics so BI teams can inter
    Alternative Data in BI: Bringing Public Web Data Into Your Dashboards
    23 Min Read
    Flat editorial illustration: The article contrasts passive follower totals with a behavioral path from attention and trust to cli
    Creator Analytics Beyond Follower Count: Seven Metrics That Measure Audience Behavior
    11 Min Read
    Flat editorial illustration: The article connects AI and marketing analytics with managerial judgment grounded in consumer behavi
    How an Online MBA in Marketing Connects AI and Data Analytics
    11 Min Read
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers -- AI-generated illustration
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers
    7 Min Read
    chatgpt image jul 21, 2026, 04 34 30 pm
    4 Core Benefits of Predictive Maintenance after Vibration Analysis
    10 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: Data Integration Processes: It’s Not the Tool, It’s How You Use It
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Big Data > Data Quality > Data Integration Processes: It’s Not the Tool, It’s How You Use It
Collaborative DataData QualityData Visualization

Data Integration Processes: It’s Not the Tool, It’s How You Use It

RickSherman
RickSherman
4 Min Read
Data Integration Processes: It’s Not the Tool, It’s How You Use It
Photo by 2427999 on Pixabay (https://pixabay.com/photos/cpu-technology-computer-3262915/)
SHARE

In my discussions with clients, prospects, students and networking with folks at seminars I am always asked about my opinion or recommendations on data integration and ETL (extract, transform and load) products. People always like to talk products and much of industry literature is centered on tools.

I’m happy to discuss products, but every once in a while someone asks me a more insightful question,  which is what happened this week. That person asked what the main shortcomings or stumbling blocks are that companies encounter when implementing data integration.

Great question. I discuss this when I am working with clients and teaching courses, but hardly anyone asks me that and directs the discussion in that direction.

My answer is simple: it’s not the tool, but how you use it that determines success. Although you do have to know the mechanics of the tool that is not the critical success factor. What really matters is the mechanics of data integration.

More Read

Data Quality – Everyone is a Stakeholder
Data Quality – Everyone is a Stakeholder
The Importance of Scope In Data Quality Efforts
How to Stay Ahead of the Data Protection Curve in 2016
New analysts and teenage love
Impressions of SAP HANA

Many people don’t understand data integration processes and the frameworks products provide to implement those processes. And it’s not just data integration newbies that have this problem; it’s also experienced veterans.

Most data integration architects, designers and developers started ETL by writing SQL scripts or manually coding using something like Java with JDBC. Then they try to replicate what they did in the manual code into the data integration processes. This is probably the worst way to use a data integration product!  You likely get little benefit from the framework, processing is not optimized (maybe even terrible) and worse, the developer gets frustrated because he feels he could have coded it faster.

Welcome to the world of frustrated data integration processes, where people either assume these products are not useful or that the particular product they used must not be very good.

Almost all data integration products provide data imports/exports; data and workflows; data transformations; error handling; monitoring; performance tuning; and many processes that have evolved as best practices such as slowly changing dimensions (SCD), change data capture (SCD),  and hierarchy management.  All of these pre-built capabilities mean that data integration development does not have to reinvent the wheel, but can leverage industry best practices to develop world-class integration. But instead many data integration developers are spending their time creating the equivalent of manually coded import, extract and transforms without ever having time to get the best practices that would best serve their business.

Any successful, productive, robust data integration effort needs people who understand the necessary processes and can implement best practices.  Getting the tool and having the people who know how to use the tool is only the beginning.  You will get nowhere fast until you make sure you have people who understand data integration processes.

TAGGED:data integration
Share This Article
Facebook Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

Flat editorial illustration: The article is about joining public web signals with internal business metrics so BI teams can inter
Alternative Data in BI: Bringing Public Web Data Into Your Dashboards
Big Data Business Intelligence Exclusive Web Analytics
Flat editorial illustration: A SaaS-editorial scene showing product usage, support patterns, and renewal feedback informing a fle
Using Product Usage Analytics and Support Data to Build a B2B Product Roadmap
Business Intelligence Infographic
From Hiring To Payroll: Managing HR Data Across Business Systems -- AI-generated illustration
From Hiring To Payroll: Managing HR Data Across Business Systems
Big Data Business Intelligence Infographic
Flat editorial illustration: The article is about automating the collection, preparation, and delivery of marketing data across p
From Data to Dashboards: 5 Marketing Reporting Automation Software Picks
Big Data Business Intelligence Exclusive

Stay Connected

1.2KFollowersLike
33.7KFollowersFollow
222FollowersPin

You Might also Like

Cloud-Based BI Dramatically Improves Collaboration
Big DataBusiness IntelligenceCloud ComputingCollaborative DataData Management

Cloud-Based BI Dramatically Improves Collaboration

3 Min Read
Informatica's Master Data Management Strategy
Big DataBusiness IntelligenceData ManagementInside CompaniesITNews

Informatica’s Master Data Management Strategy

3 Min Read
Data-Driven BPM: Making "Big Data" Actionable
Big DataCloud ComputingData ManagementData MiningData WarehousingKnowledge Management

Data-Driven BPM: Making “Big Data” Actionable

4 Min Read
Flat editorial illustration: A data integration ecosystem showing the six interconnected components (sources, big data storage, d
AnalyticsBig DataData ManagementExclusiveIT

Data Integration Ecosystem for Big Data and Analytics

10 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

5 Great Tips for Using Data Analytics for Website UX
5 Great Tips for Using Data Analytics for Website UX
Big Data
ai chatbot
How AI Website Chatbots Improve Customer Support and Lead Generation
Chatbots Exclusive

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-26 SmartData Collective. All Rights Reserved.
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?