Question Assumptions Before Initiating Big Data Projects

If you ever undertook the task of building a house on an empty plot of land there a number of factors that you would want to consider.

Before you even start to lay the important foundations you would want to undertake a survey of the Ground Quality. This would typical involve a bore hole analysis to assess the type and quality of the ground. Alongside the bore hole analysis you’d want to analyse the landscape of the plot. For example, are there tree roots close to the planned structure that could impact foundations? Does the plot have sufficient access to required services such as water and sewerage?

In the world of construction, if you want to ensure that you are building something which will be successful, have longevity and contain no hidden surprises you would ensure that the ground work is undertaken in advance.

In Data Migrations, CRM Implementations and Data Warehouse projects you often read assumptions in project documents such as ‘the data is assumed to be fit for purpose and adhering to the relevant business standards’. How often is this assumption found to be incorrect, leading to delayed project completion or poor user adoption?

An equivalent of the bore hole analysis would be a data profiling exercise that ascertained what the key data items were, what good quality data looks like and how data performs against these expectations.

An equivalent to a landscape analysis would be to ensure that the system architecture can support both current and future demands, that the right people are in place, and that any required change could be easily undertaken.

These items are key components to a successful implementation but all too often they do not get the time that they deserve on the project plan.

Why are we not taking the opportunity during these transformational projects to question the data and architecture that is relied upon to make the project successful? Why do we all too often assume?