The BI Survey is created each year by analyst Nigel Pendse of the Business Application Research Center (BARC) headquartered in Germany. BARC was founded in 1994 as a spin-off from the University of Würzburg’s Chair for Information Science.
The BI Survey is based on the world’s largest independent survey of Business Intelligence and Performance Management users. For this year’s report over 2600 respondents (1894 users, 256 consultants and 472 vendors) from around the world took part. I have been lucky enough to have been given a review copy and on reading it I have had two reactions:
- This is a BIG report with over 550 pages, 350 figures and is a 34 MB pdf file.
- It lives up to the author’s claims to be comprehensive, insightful and practical.
This report is so comprehensive that it is difficult to give you a fair summary of its content. Here are a couple of details to give you an idea.
The products covered are: MicroStrategy, Infor PM OLAP, QlikTech QlikView, Board, Microsoft Analysis Services, Cognos Reporting, BusinessObjects, SAP BI/BW, WebFOCUS, Cubeware Cockpit, Cognos TM1 Server, arcplan, Microsoft Reporting Services, Panorama NovaView, Bissantz, …
The BI Survey is created each year by analyst Nigel Pendse of the Business Application Research Center (BARC) headquartered in Germany. BARC was founded in 1994 as a spin-off from the University of Würzburg’s Chair for Information Science.
The BI Survey is based on the world’s largest independent survey of Business Intelligence and Performance Management users. For this year’s report over 2600 respondents (1894 users, 256 consultants and 472 vendors) from around the world took part. I have been lucky enough to have been given a review copy and on reading it I have had two reactions:
- This is a BIG report with over 550 pages, 350 figures and is a 34 MB pdf file.
- It lives up to the author’s claims to be comprehensive, insightful and practical.
This report is so comprehensive that it is difficult to give you a fair summary of its content. Here are a couple of details to give you an idea.
The products covered are: MicroStrategy, Infor PM OLAP, QlikTech QlikView, Board, Microsoft Analysis Services, Cognos Reporting, BusinessObjects, SAP BI/BW, WebFOCUS, Cubeware Cockpit, Cognos TM1 Server, arcplan, Microsoft Reporting Services, Panorama NovaView, Bissantz, Hyperion Essbase, Crystal Reports, Cognos Analysis, Targit, Oracle BIEE/BISEO, Actuate Platform, Microsoft Excel PivotTables, MIK, LogiXML, Oracle Discoverer, SAS, Palo, Hyperion Interactive Reporting, Microsoft PPS, ProClarity or BSM, Other Business Objects product, CyberQuery, Other Cognos product, Oracle OLAP Option, Hyperion Planning, SAP BPC (OutlookSoft), Hyperion Financial Management.
There are of course many hundreds of other tools out there that are not included. This report targets the top 30 – 40 globally. Take a look at the CORTEX forum on local vendors for examples.
Here are the major sections of the report:
- Introduction1
- The sample
- Products included
- Age Profiles
- The Business Benefits Index
- The purchase cycle
- The BI ownership experience
- Vendor effectiveness
- Implementation
- Timescales
- What goes wrong?
- Applications
- Web BI
- Server platforms
- Client/server combos
- Source databases
- Data Volumes
- Performance at the speed of thought?
- The customers’ verdict dashboards
- Appendix: Survey questionnaire
Each section contains extensive and useful (i.e. insightful) analysis from the survey.
The report is not cheap at US$5,000, but if I was in the market to buy a BI platform then the cost would not be a barrier. After all what are the alternatives? I don’t know of any, although options often taken are:
- Employ a consultant – directly or through a firm like Accenture, Deloittes, etc. You would never get someone with recent experience of all 35+ products included in the BI Survey. In my defence (as a working consultant), I believe that the BI Survey (facts and insight) and a consultant (local knowledge and experience) can be a winning combination.
- Subscribe to Gartner or Forrester to get access to their research. They certainly publish some good stuff but it can be a little light on detail unless you pay a lot of dollars for the detailed industry analysis. Either way, it will cost a lot more than the 5K for the BI Survey. This is not really a fair comparison as they are addressing different needs. The BI Survey is focused on an investment decision. Gartner and Forrester both offer a stream of research that covers many more business needs2.
- Do the research yourself and speak to the vendors. You will need to do this anyway as part of your normal purchasing process. Trusting vendors exclusively to advise you is a brave strategy.
It is a little easier to say why the BI Survey report could be useful to anyone making BI related investments as the report gives you:
- Detailed evaluations (and sometimes pricing) of all the major BI tools.
- Deep perspective on how the tools are being used by your peers and across different industries.
- Customer Verdicts that provide a quick snapshot of how a particular product was rated in the last edition of The BI Survey. 23 products are assessed across a set of 26 normalized root KPIs (see a sample below). This is a new feature and I hope to see it retained in subsequent surveys.
So you can tell I like the report. Are there any ways to improve it? Well yes, of course there are. Here are my suggestions for the 2009 BI Survey:
- Include more on open source and cloud (SaaS) based solutions. The authors of the report note that they found it “very hard to find users of these tools, even with the help of the vendors (Actuate, JasperSoft, Jedox and Pentaho)” I am surprised. The authors go on the observe that “It may be that open source BI components are mainly embedded in other applications, so the end users may not even be aware that they are using them.” Elsewhere in the report is included an open source database BI league table. I look forward to more detail being included in the next annual report.
- Price guidance is a current gap. The report does include pages on a Cost of Ownership Index and an analysis of the types of licenses purchased but more information would be useful. Anything that can provide more hard details on the total cost of ownership of each tool would be a valuable additional feature. Difficult for BARC to do perhaps. As mentioned, there are insights given into license fees – I just want more!
- Drop the practice of using vendors to inform their customers of the survey, as this could lead to skewed results if the vendor decided only to inform customers they knew were happy. The temptation for vendors to not inform unhappy customers is too great.
- Increase SME participation. Small, medium and large companies are all well represented and I would like to have more insight into how smaller and medium sized enterprises differ from the big end of town.
The 2009 Survey is now open and if you are a BI user and want to complete the survey then you can go online here. I encourage you to do it as it should only take about 15-20 minutes to complete.
1Includes a good 20-page executive summary.
2BARC also publishes the The BI Verdict (the new name for The OLAP Report).
