Data Gazers

Within cubicles randomly dispersed throughout the sprawling office space of companies large and small, there exist countless unsung heroes of enterprise information initiatives. Although their job titles might be labeling them as a Business Analyst, Programmer Analyst, Account Specialist or Application Developer, their true vocation is a far more noble calling.

They are Data Gazers.

In his excellent book Data Quality Assessment, Arkady Maydanchik explains that:

“Data gazing involves looking at the data and trying to reconstruct a story behind these data. Following the real story helps identify parameters about what might or might not have happened and how to design data quality rules to verify these parameters. Data gazing mostly uses deduction and common sense.”

All enterprise information initiatives are complex endeavors and data quality projects are certainly no exception. Success requires people taking on the challenge united by collaboration, guided by an effective methodology, and implementing a solution using powerful technology.

But the complexity of the project can sometimes work against your best intentions. It is easy to get pulled into the mechanics of..…

They are Data Gazers.

In his excellent book Data Quality Assessment, Arkady Maydanchik explains that:

“Data gazing involves looking at the data and trying to reconstruct a story behind these data. Following the real story helps identify parameters about what might or might not have happened and how to design data quality rules to verify these parameters. Data gazing mostly uses deduction and common sense.”

But the complexity of the project can sometimes work against your best intentions. It is easy to get pulled into the mechanics of documenting the business requirements and functional specifications and then charging ahead on the common mantra:

“We planned the work, now we work the plan.”

Once the project achieves some momentum, it can take on a life of its own and the focus becomes more and more about making progress against the tasks in the project plan, and less and less on the project’s actual goal… improving the quality of the data.

In fact, I have often observed the bizarre phenomenon where as a project “progresses” it tends to get further and further away from the people who use the data on a daily basis.

However, Arkady Maydanchik explains that:

“Nobody knows the data better than the users. Unknown to the big bosses, the people in the trenches are measuring data quality every day. And while they rarely can give a comprehensive picture, each one of them has encountered certain data problems and developed standard routines to look for them. Talking to the users never fails to yield otherwise unknown data quality rules with many data errors.”

There is a general tendency to consider that working directly with the users and the data during application development can only be disruptive to the project’s progress. There can be a quiet comfort and joy in simply developing off of documentation and letting the interaction with the users and the data wait until the project plan indicates that user acceptance testing begins.

The project team can convince themselves that the documented business requirements and functional specifications are suitable surrogates for the direct knowledge of the data that users possess. It is easy to believe that these documents tell you what the data is and what the rules are for improving the quality of the data.

Therefore, although ignoring the users and the data until user acceptance testing begins may be a good way to keep a data quality project on schedule, you will only be delaying the project’s inevitable failure because as all data gazers know and as my mentor Morpheus taught me: