Data Quality Terms Defined

After 20 years developing data quality solutions, the team at DQ Global are very familiar with data quality terminology. However, many customers tell us that data quality terms can be confusing. What is the difference between enrichment and enhancement?

After 20 years developing data quality solutions, the team at DQ Global are very familiar with data quality terminology. However, many customers tell us that data quality terms can be confusing. What is the difference between enrichment and enhancement? And why do we cross match or automate in our quest for clean data?

In this article, we’ll define some of the terminology we use on a daily basis to describe our data quality solutions.

Deduplication and Cross Matching: What’s the Difference?

‘Deduplication’ and ‘cross matching’ are often used in the same sentence, particularly in reference to DQ Global’s Match software. But they are not the same thing: they complement each other.

Our Match application is technically a deduplication tool in that it locates duplicate records in a database or an application. Match applies various methods to deal with those duplicates. It may suppress a duplicate record, merge it with another record, or flag it up for manual review.

Cross matching is slightly different. In medicine, you will hear the term used to describe a process of finding compatibility. For example, cross matching may be used when an organ donor and potential organ recipient are identified.

In data terms, cross matching means scanning separate datasets and finding matches based on common fields. It is a flexible way of finding a duplicate, even if the two records are not exactly the same. In criminology, cross matching is sometimes used to identify people when only a subset of information is known, making it suitable for fuzzy matches.

In your data quality process, cross matching will be used to compare different datasets and locate matches laterally between them. Cross matching can locate the same contact record in multiple formats, helping you to build a picture of the customer by deduplicating and merging records and building on what you have.

If successful, persistent and efficient, this process leads to the Single Customer View (SCV), a ‘master database’ containing high quality records on all of the contacts you have.

Data Enrichment or Data Enhancement?

As part of a data quality initiative, DQ Global can enhance a database record. This involves adding supplementary information, such as demographic data, that aids business decisions and makes the dataset more useful and versatile. Enhancement is an extension of the data that already exists, and it normally involves importing data from third party databases.

Data enrichment involves fortifying the data; adding value to it with intelligent analysis. This encompasses enhancement, which we’ve already mentioned, as well as other techniques. These may be simple – for example, applying likely spelling corrections – or complex. Often, enrichment means applying algorithms to enhanced data to find unlikely combinations of fields, or extrapolating the data to better understand customer behaviour.

Businesses enrich data to find out what kinds of decisions their clients would make. Things like political affiliations, disposable income and preferred communication methods all help your staff to make better decisions. In effect, data enhancement is a building block in enrichment. By using a combination of methods, our solutions find the ‘bigger picture’, using your data as the foundation.

Migration and Integration

Data migration and data integration are two processes that have features in common, but are actually very different in practice.

Data migration involves moving data from one system to another, in one direction. Migration is a one-time process. Once data has been migrated, it is not moved back, and the migration is not repeated.

Integration is the meshing of two systems that do not already talk to each other. Integration is repeatable, too. Often, it means creating a two-way link so that users see a more complete picture of a record or contact. Integration is commonly used in cloud applications: for example, your Customer Relationship Management (CRM) system and your accounting tool may be linked so that you can see invoices, contact details and payment history in both.

Automation

The final link in the chain is business automation. Automation describes the process of converting a manual sequence of steps into a sequence that is triggered automatically by a computer, or with a single human ‘click’ of the mouse.

Why bother with automation? Businesses often struggle with repeated manual tasks: they tend to be dull for employees, and they drain teams of enthusiasm and resources. Repetitive tasks are a prime source of data quality problems: data is often submitted in an incomplete state, with errors, with inaccurate or invalid data in place.

Often, manual processes have been adopted because the software in use is too old to be compatible with anything else, yet business automation can help old software ‘talk’ to new applications.

By using process automation, many data quality challenges are overcome. For example, manual data entry is no longer the source of typos and spelling mistakes. Workflows run from beginning to end, and any data quality issues can be ‘ironed out’ as the workflow completes.

Automation also makes processing much faster. The automation can also be run unattended at a time or event of your choice. Workflows are created and managed using a drag and drop interface that anyone can get to grips with.