By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData Collective
  • Analytics
    AnalyticsShow More
    data analytics in sports industry
    Here’s How Data Analytics In Sports Is Changing The Game
    6 Min Read
    data analytics on nursing career
    Advances in Data Analytics Are Rapidly Transforming Nursing
    8 Min Read
    data analytics reveals the benefits of MBA
    Data Analytics Technology Proves Benefits of an MBA
    9 Min Read
    data-driven image seo
    Data Analytics Helps Marketers Substantially Boost Image SEO
    8 Min Read
    construction analytics
    5 Benefits of Analytics to Manage Commercial Construction
    5 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: A more appropriate metaphor for business intelligence projects
Share
Notification Show More
Latest News
data analytics in sports industry
Here’s How Data Analytics In Sports Is Changing The Game
Big Data
data analytics on nursing career
Advances in Data Analytics Are Rapidly Transforming Nursing
Analytics
data analytics reveals the benefits of MBA
Data Analytics Technology Proves Benefits of an MBA
Analytics
anti-spoofing tips
Anti-Spoofing is Crucial for Data-Driven Businesses
Security
ai in software development
3 AI-Based Strategies to Develop Software in Uncertain Times
Software
Aa
SmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Big Data > Data Warehousing > A more appropriate metaphor for business intelligence projects
Business IntelligenceData Warehousing

A more appropriate metaphor for business intelligence projects

Peter James Thomas
Last updated: 2009/03/18 at 1:47 PM
Peter James Thomas
10 Min Read
SHARE

A traditional metaphor for IT projects

A traditional metaphor for IT projects

IT people are familiar with a number of metaphors for their projects. The most typical relates to building; IT projects are compared to erecting a skyscraper. The IT literature is suffused with language derived from this metaphor. We build systems. We develop blueprints for them. We design architectures (two-for-one there). This analogy has some strength and there are indeed superficial similarities between the two areas. However, as with most metaphors, if over-extended their applicability often breaks down. I recall one CEO in particular who was obsessed by the “building team” moving on to the next “site”; regardless of the current one requiring further work and dedicated maintenance. One of his predecessors often referred to wanting a “diesel submarine” built, as opposed to a “nuclear one”. Before I fall into the same trap of over-exploiting the metaphor, let’s move hurriedly on.

More Read

BI and analytics

The Role of Analytics and BI in the Entertainment Industry

How To Enhance Your Jira Experience With Power BI
10 Best Practices For Business Intelligence Dashboards
6 Valuable Business Intelligence Lessons From Brexit
6 Ways To Raise Your Business Intelligence To The Next Level

As I mention above, aside from the occasional misapplication, the building analogy works reasonably well for many IT projects; does it also work for business intelligence? I think that there are some problems in applying the metaphor… 

A traditional metaphor for IT projects

A traditional metaphor for IT projects

IT people are familiar with a number of metaphors for their projects. The most typical relates to building; IT projects are compared to erecting a skyscraper. The IT literature is suffused with language derived from this metaphor. We build systems. We develop blueprints for them. We design architectures (two-for-one there). This analogy has some strength and there are indeed superficial similarities between the two areas. However, as with most metaphors, if over-extended their applicability often breaks down. I recall one CEO in particular who was obsessed by the “building team” moving on to the next “site”; regardless of the current one requiring further work and dedicated maintenance. One of his predecessors often referred to wanting a “diesel submarine” built, as opposed to a “nuclear one”. Before I fall into the same trap of over-exploiting the metaphor, let’s move hurriedly on.

As I mention above, aside from the occasional misapplication, the building analogy works reasonably well for many IT projects; does it also work for business intelligence? I think that there are some problems in applying the metaphor. Building tends to follow a waterfall project plan (as do many IT projects). Of course there may be some iterations, even many of them, but the idea is that the project is made up of discrete base-level tasks whose duration can be estimated with a degree of accuracy. Examples of such a task might include writing a functional specification, developing a specific code module, or performing integration testing between two sub-systems. Adding up all the base-level tasks and the rates of the people involved gets you a cost estimate. Working out the dependencies between the base-level tasks gets you an overall duration estimate.

The problem with BI projects is that some of the base-level tasks are a bit different. An example might be: develop an understanding of a legacy data table, how it relates to other legacy data sets and to more modern systems (this sits under area two of my model of BI development – see BI implementations are like icebergs). This is not an exercise that is very easy to estimate in advance. Indeed it may not be possible to produce an adequate estimate until a substantial amount of work has been done. Even at a late stage in the task, something may be discovered which expands the work required dramatically; surprises may lurk round every corner.

Why is this? Well with legacy data, the people who developed the system may have done so many years ago. Since then, they may have left the company or moved on to other areas, taking their knowledge with them. Their place may have been taken by successive tranches of new staff. Perhaps poor initial documentation meant that later workers did not fully understand the full nature of the system, but nevertheless did their best to build upon it. Perhaps the documentation was good at first, but has not been kept up-to-date and now describes a system that no longer exists.

By definition, legacy systems will have been around for some time and layers of changes will have accumulated on top of each other. Maybe, as a company has expanded, new data has been interfaced to the system from different business units and territories; perhaps each of these cases has its own dedicated interface code, each subtly different from those of other systems. Even where people exist in an organisation who preserve an “oral tradition” about the system, handed down to them over generations; these people may not appreciate how their data interacts with other data – even if the person who looks after another legacy system sits in the adjacent cubicle.

Waterfall Project Plan (intentionally blurred somewhat)

Although these challenges can also occur when trying to understand the data in more modern systems, they are particularly acute with older ones. For a start, the people who designed these systems are more likely to be around. Also legacy systems often sit at the centre of a Byzantine web of inter-connections, batch-processes, over-night jobs and the occasional more modern service. It can be a real mess and this is a situation with which that any data analyst with a reasonable amount of experience will be very familiar.

The difficulty of estimating the duration of tasks such as properly analysing legacy data makes overall estimation of BI projects more of an art than a science. Of course techniques such as time-boxing tasks can be applied, but these are not always 100% appropriate. A 75% analysed data source (even assuming that the estimate that only 25% work is left is accurate) is not an analysed data source. Leaving dark corners of knowledge is likely to be reflected in BI cubes and reports that do not reconcile back to their sources. Probably the best way to deal with this problem is to be extremely open about the challenges up-front with executive sponsors and when submitting estimates. It helps to also stress the level of uncertainty in progress reports. The more honest you are initially, the better you will be able to explain any overruns and the more likely it is that you will be believed.

These types of issues mean that the – hopefully more orderly – process of constructing a building is not a fully accurate way to describe a BI project. That is unless the metaphor is extended to include an occurance that is all too common during construction in The City of London. Given the age of Londinium, whenever ground is broken on a new project, it is more likely than not that a mediaeval, Anglo-Saxon or Roman site is unearthed (often all three). These finds, while of enormous interest to academics, can result in projects being put on hold (sometimes for years) while the dig is fully assessed, artefacts are carefully removed and catalogued by experts and so on. Sometimes the remains are of such importance that a structure preserving and protecting them (and even allowing public viewing) has to be made part of the design of the foundations of new building. Many office blocks in The City have such viewing galleries in their basements. Such eventualities can create massive and unexpected overruns in central London building projects.

So this leads me to suggest a different metaphor for BI projects. Major elements of them are much more like archaeological digs than traditional building. The extent and importance of a dig is very difficult to ascertain before work starts and both may change during the course of a project. It is not atypical that an older site is discovered underneath an initial dig, doubling the amount of work required.

So, my belief is that BI professionals should not be likened to architects or structural engineers. Instead the epithet of archaeologist is much more appropriate. And if the fedora fits, wear it!

Fortune and glory in BI?

Fortune and glory in BI?

 

Bookmark this article with:
Technorati| del.icio.us| digg| Reddit| NewsVine

 

Posted in business, business intelligence, management information, project management, technology Tagged: bi, business intelligence, information technology, it management, it projects, la, legacy data, legacy systems, management information

TAGGED: bi, it projects
Peter James Thomas March 18, 2009
Share this Article
Facebook Twitter Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

data analytics in sports industry
Here’s How Data Analytics In Sports Is Changing The Game
Big Data
data analytics on nursing career
Advances in Data Analytics Are Rapidly Transforming Nursing
Analytics
data analytics reveals the benefits of MBA
Data Analytics Technology Proves Benefits of an MBA
Analytics
anti-spoofing tips
Anti-Spoofing is Crucial for Data-Driven Businesses
Security

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

BI and analytics
Analytics

The Role of Analytics and BI in the Entertainment Industry

5 Min Read
power BI solutions
Business Intelligence

How To Enhance Your Jira Experience With Power BI

12 Min Read
business intelligence tools
Best PracticesBusiness IntelligenceExclusive

10 Best Practices For Business Intelligence Dashboards

9 Min Read
business intelligence lessons from Brexit
Business IntelligenceBusiness RulesExclusive

6 Valuable Business Intelligence Lessons From Brexit

7 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

AI and chatbots
Chatbots and SEO: How Can Chatbots Improve Your SEO Ranking?
Artificial Intelligence Chatbots Exclusive
ai in ecommerce
Artificial Intelligence for eCommerce: A Closer Look
Artificial Intelligence

Quick Link

  • About
  • Contact
  • Privacy
Follow US

© 2008-23 SmartData Collective. All Rights Reserved.

Removed from reading list

Undo
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?