Introduction
Imagine that you’re at the very end of a long development process of a BI solution for internal users within your organization. You might have the feeling that once you release your solution to its consumers your job will be easier and less demanding. Well, that might not be the case.
The development stage is usually very demanding and can consume a huge amount of time and effort till you release the first version of your solution. However, it is only after the release that users actually start using the solution and experience their theoretical requirements as an actual analytical tool in their day to day work – which is when a host of new complications appear.
There are few scenarios that can occur over time:
- The worst case: your users do not adopt your solution for some reason (bad implementation, not aligned to the core needs of your users, natural adoption pains, etc.) In this case, you’ll need to apply a lot of changes or even start over.
- The best case: your users find your solution extremely useful, it is extremely beneficial to your company and it is in focus of senior management discussions. Naturally, it will open up the appetite of your users and stakeholders and new feature requests and change requests will start flowing in your direction.
- Anywhere in between.
Either way, you’ll need to be ready to implement changes and upgrades to your solution on an ongoing basis.
Your Customers and You
You can think of it as managing a product\service such as the one your company provides to your customers. And while you’re at it, you can think of your end users and stakeholders as your (internal) customers who have to be satisfied with the product you are “selling” them.
As long as you live by this approach and keep in mind that your product lives, breathes and has to be managed to keep it beneficial to your organization – your life as a BI product owner will be easier.
In this guide, I will describe some aspects of the delivery of the BI solution throughout various stages in its life cycle: change management, successful rollout, data reliability and more. I will mainly focus on internal BI solutions for medium-large organizations, although most parts may be relevant for other scenarios as well. We will focus on the business aspects of this process rather than the technical ones.
3 Keys to Success
1. Plan and Govern
In order to be ready for the next requirement\release\fix, you need to be prepared. Here are a few areas you should consider while dealing with changes before you go live:
Ensure Data Reliability and Gain Customer Confidence
We need to keep in mind that the main purpose of having a BI solution is for the organization to make informed decisions based on the information displayed in the dashboards (KPIs and charts). If the data isn’t accurate, you’ve failed right out of the gate.
Centralizing your data in a data mart or data warehouse that serves as your single source of truth (SSOT), and consolidates the data from all relevant sources, definitely helps. However, in some cases this won’t be enough to ensure that the data is always reliable and consistent:
- If calculations are made on the front-end there might be several models with different metrics with the same KPI terminology. E.g., in one dashboard revenue is being calculated according to the currency rate at the beginning of the month and on another, it is calculated based on the rate at the end of the month.
- In big enterprises, there can be more than one data warehouse with different calculation methods and definitions.
- In some cases, there could be inaccuracies in the data sources themselves (garbage in – garbage out).
- For the dashboards to serve the organization’s missions it must be clear what is behind the numbers. When a measure is too complex and requires elaborations time after time it misses this objective.
To address consistency and reliability issues, consider having one place that will gather all KPI definitions and logic AND will be accessible to all end users, stakeholders and occasional viewers.
You might consider having a reference to these explanations from the dashboard itself (using tooltips or links to online documentation).
Know Your Users and Stakeholders’ Roles and Responsibilities
In every point in time but especially prior to the first release, you must become very familiar with your audience. Learn who are the decision makers that will most benefit from your product, who are your heavy users, who are most familiar with the business processes and needs and who are the ones that will be most cooperative (and, conversely, who are the ones that will hold you back).
To understand what types of permissions, roles and responsibilities each user of the BI solution will have, create an organizational map with all the relevant information before you deliver the solution.
Once you map out all the relevant stakeholders, taking their roles and responsibilities into account, it will become much easier to determine the role each one of them will have in your BI solution – who set priorities, who can provide meaningful feedback, etc. It should look something like this:
Define operational methods:
Define the delivery method, working processes and standards, and data control processes for your BI product, per the organization’s culture and the goals of the BI initiative.
You’ll want to choose the right delivery method for your organizational culture – there are two fundamental approaches:
- Centralized – deliver a full solution that contains all back end and analytical components according to the business requirements, with no room for adjustments by the end user. This approach is more suitable for a business user that knows what he\she wants to see, and the business process that is being analyzed is quite stable and doesn’t change frequently. In this case, you’ll need to put more effort on the training part and documentation.
- Decentralized – Meaning you deliver an initial, “vanilla” solution that is open for adjustments. This approach is more suitable for dashboard designers that are from the business side and not part of your BI team, and also advanced analysts that need to create ad hoc reports and to use different analytic tools per the case they want to analyze during their day to day. In this case, you should place more emphasis on short release cycles that will include your advanced users’ feedbacks as much as possible. You can also consider certifying these users for better control of the product’s configuration.
In most cases, some kind of middle-ground approach between these two ends will be the best fit. The question is which side of the scale your organizations leans towards, and to what extent.
Understanding the delivery method will help you choose how to focus your efforts – whether to design and implement an end-to-end solution that caters to end users, or to focus more on the back-end and creating a strong governance and training plan that caters to dashboard designers and analysts.
You’ll also want to set working processes and standards that will enable collaboration between developers and designers if needed.
Finally, you should create a control process for data sources to ensure their reliability over time:
- In some cases, the BI product owner would be an accountable factor, and his or her signoff would be required prior to any change in an asset that relates to his\her data source.
- Have a weekly\monthly sync with the DBA or data source owner.
Change Management – Adopting a Version Release Strategy:
Just like your company’s product changes based on its customers’ needs and preferences, so does your BI product. During its lifecycle you’ll need to refer to change requests, new feature requests, bugs or calculation issues arise.
Here are a few points to address for an efficient implementation process:
Backlog Prioritization – some of the requests are more urgent than the others and some are more important. Part of the release strategy is to prioritize them according to your organizational culture. For each feature or user story, you might consider its overall effect on the company, if it will reduce expenses, save time or create new growth opportunities. You can also ask if the new request is aligned with the vision and strategy of your team, department, or organization and measure its ROI according to your effort estimation. You might also consider its contribution to the robustness of the solution (which is often the case with internal requests from the development team to add new features or improve performance).
Backlog prioritization should be transparent and in cooperation with all relevant stakeholders, who should understand the reasoning behind your decisions.
Versioning – all changes that you apply should be available to your users. To have better control of these changes, you would generally want to implement some form of versioning system so that each release of a new version contains a bucket of changes.
Release cycles, change requests and hotfixes: Short release cycles could mean leaner and less substantial updates. It’s a tradeoff between a fast go-to-market approach vs bigger and more meaningful releases.
You can use any combination of short and long release cycles to best serve your customers, but it has to be consistent to align expectations for new features (a release every month, quarter, three weeks…)
You can decide which features will be included in the next release according to one of the many software management methods and backlog prioritization approaches that are available while considering their priority and effort (e.g. Agile, Kanban).
Large changes may take longer than your typical cycle period.
Let’s say you decided on version releases on a bi-weekly basis and you have a change request that will take 6 weeks to develop. In this case – and actually for any feature – you might want to divide it into smaller stories (when possible) that could more easily be split between future releases. If this is not an option than work on it in parallel to the ongoing releases and once ready, add to the next release.
For an extremely urgent and important set of fixes, you can release a hotfix version. The criteria for it should be known in advance.
Documentation – every release changes your product to some extent (more capabilities, different calculation mode, new KPIs, etc.)
In addition to communicating these changes, there should be one place that will keep on tracking the changes of each release (for both internal use of the development team and the external use of the dashboard consumers).
2. Plan the rollout
The rollout phase starts once the development phase is done.
A successful rollout is one of the most important ingredients to gain engagement from users in the organization, as you’ll never get a second chance to make a first impression.
A successful rollout should include:
- User acceptance testing (UAT) and sign-off from each relevant stakeholder. This will ensure that all involved will also be committed and engaged with the product as it rolls out.
- Freeze the current phase of development in the staging environment until after the production environment is fully live.
- Launch to your target audience.
- Provide a short description of the dashboard, its objective and how to use it. Can be communicated by email, tooltip, department meetings or other methods.
- Onboarding – In addition to the release notes a training program must be in place (as described below)
- Suggest close support for few weeks after the launch.
You can also think of a gradual release where the more savvy users will be the first to use your solution (as a beta version of sorts), and after getting their feedback you can continue with the release for all other users (or continue with the gradual release according to its relevance to the customer – first the ones that are dealing with the data on a daily basis, then their managers and finally senior management).
The onboarding process
Even if your users are familiar with the technology and platform you are using, your specific solution is new for everyone when it is first released and there might be features that are not completely intuitive.
In addition to the release notes, you should consider having a training program in place (especially for the first release).
In addition, during the life cycle of your product, new users will probably join the circle. You can think about having personal training sessions, recorded sessions or some online tutorials and documentation that will help your new users get to know your product and deepen their knowledge over time.
3. Push for adoption
What do you want to measure?
The main KPIs for a typical product release are profit, market share, and growth. These metrics tell us if our product is successful or not and whether customers respond positively to it. Since your BI product can’t be measured this way, you need to think about other success criteria to know if you on the right path, if you need to act differently, and to proactively apply changes and learn from this experience towards the next release.
Success criteria should be based on the impact that the business intelligence system had on the business, the level of use among your potential users (# of users, frequency of use) and the level of awareness and involvement within the organization.
Here are some examples of questions that can help you determine your success criteria:
- Benefits:
Cutting costs – How much money can I save by using the BI product? Case in point, by exposing and reducing operational and financial waste, improving processes, identifying new business opportunities by consolidating several data sources and revealing a full picture of a particular business scenario
How many working hours are being saved per month by replacing manual ETL or other procedures?
- Relevancy to the business requirements and the product’s ease of use – How many users are using the BI solution every month? What is their rate among all potential users? Can you see a growth in usage in the last few weeks? What is their frequency of use (monthly\weekly\daily\hourly per business factor)? Does the solution help them deal with their day-to-day work? With making an important decision? Helping them meet their personal targets? The company’s target?
Smart Ways to Measure Success
Feedback is a gift you receive from your users, so don’t pass on it.
Once you have determined your KPIs, the next question is – how do you measure them?
Stay in the loop: To keep the solution beneficial to the organization you have to track the satisfaction of your users from the product, understand what results they are seeing, what can be improved and be the first one to know about any issue your end users are dealing with.
To do so you can use some of the following methods:
- Have a shared list for suggestions for improvements.
- Implement an in-product feedback system that will allow you to get feedback in real time.
- Meet with midlevel and senior management.
- Join relevant department meetings to gather feedback.
Analyze your analytics – if you can reach the metadata of your product usage, you can analyze what areas work better than others and take actions accordingly. You can discover more about this option in this article.
It’s your job to spread the word!
You want to engage new potential users in using your product and increasing the benefits it delivers to your organization. So go ahead and make some noise! This can be done through:
- Periodic communication.
- Success stories: successful releases, organizational benefits that can be related directly to your product, before and after stories, user quotes, etc.
- Present a flow within the dashboard which demonstrate the relevant business processes to the specific audience.
- Centralized training days. In addition to the obvious benefit of having users to be more self-sufficient, it can also create a buzz in the organization.
- Present the solution to new business units.
- Send samples of the specific dashboard with an executive summary to introduce the solution to the new potential users.
Summary
Delivery is a crucial stage in every business intelligence solution, and you should plan for it while you’re planning of the solution itself and before you start development. Plan the delivery approach you think will be most effective for your organization. Treating your BI solution as an internal product with a long life, and one that has a tangible and important value proposition will help you implement your solution as a core part of your company’s toolset and a highly effective means of improving its operations.