The Federal Government and Analytics

Steve Bennett
Posted by July 13, 2009
I have recently read the latest Australian Federal Government’s report on innovation. The report is called ‘Powering Ideas: An Innovation Agenda For The 21st Century‘. It caught my eye as I was hoping to learn of a new impetuous for analytics. Well it was a fairly interesting read and our field did get a single mention – but interestingly only as a way of measuring the success of the innovation policy itself. There is no concept of using analytics to directly improve an organisation’s performance, or fuel greater innovation of new products and services.

From page 29 of the report:

Measurement
is the only way
to be sure that
policies are
working and
resources are
being put to the
best possible use.

Next to this declaration is the only paragraph that mentions analytics: 

“The Australian Government has a duty to measure the impact of
specific innovation initiatives and the performance of the system as
a whole. This is the only way to be sure that policies are working and
resources are being put to the best possible use. Indicator analysis,
scorecards, and case studies are all commonly used for this purpose,
but they only tell part of the story. Econometric analysis has the
potential to tell us much . …

I have recently read the latest Australian Federal Government’s report on innovation. The report is called ‘Powering Ideas: An Innovation Agenda For The 21st Century‘. It caught my eye as I was hoping to learn of a new impetuous for analytics. Well it was a fairly interesting read and our field did get a single mention – but interestingly only as a way of measuring the success of the innovation policy itself. There is no concept of using analytics to directly improve an organisation’s performance, or fuel greater innovation of new products and services.

From page 29 of the report:

Measurement
is the only way
to be sure that
policies are
working and
resources are
being put to the
best possible use.

Next to this declaration is the only paragraph that mentions analytics: 

“The Australian Government has a duty to measure the impact of
specific innovation initiatives and the performance of the system as
a whole. This is the only way to be sure that policies are working and
resources are being put to the best possible use. Indicator analysis,
scorecards, and case studies are all commonly used for this purpose,
but they only tell part of the story. Econometric analysis has the
potential to tell us much more about how well the system is doing.
So do studies linking innovation data to statistics of other kinds
– financial, administrative, environmental, social and so on.
The Australian Government will work to collect better data on
innovation and develop new, more sophisticated analytical
capabilities. It will also produce an annual report on the
performance of the national innovation system. The report will
identify new opportunities and challenges, and provide regular updates
on implementation of the National Innovation Priorities.”

It’s all a bit too motherhood for me. Where are the specific criteria, KPI’s, etc. Let’s hope that they will be determined before, and not after, the funding decisions are made. I’m not holding my breath.

Interestingly, the report itself has been analysed using the innovative Leximancer text analysis tool. Leximancer is a software tool used to find meaning from qualitative data (typically text-based documents) and is a product of a small Australian company based in Brisbane. The software automatically identifies key themes, concepts and ideas by data mining large amounts of text (but not diagrams), and visually represents information in ‘concept maps’ showing the main relationships. These relationships can be examined in more detail by visually exploring major connections.

Analysis using Leximancer  was undertaken of all submissions to the Government Review that were in English, not hand-written or diagrammatical. Confidential submissions were not included. In all, 606 submissions were analysed. The maps display the frequency of the concept by the boldness of its text; the extent of the connectedness of the concepts is shown by size of concept point. The closer to the blue end of the spectrum, the more important the concept overall, reflecting frequency and connectedness. Here’s an example from the analysis:

2008 Leximancer_Submission_Analysis_dia

You can read the results of the analysis in ‘A Content Analysis Undertaken By Prof. Mark Dodgson Of The University Of Queensland, Using Leximancer Software To Identify The Main Themes, Concepts And Ideas Identified In Submissions To The Review’ here:

While (analytically speaking) the only ray of light from the report is that someone did some textual analysis I don’t despair. At least individual arms of the Government are increasingly cutting-edge. Examples include the military/intelligence community and the benefits they get from close cooperation with the US, and other agencies like the Australian Tax Office that have developed substantial and sustained analytic teams. Good news – but I have to go now and file my tax return. I have this strange feeling that someone is watching me!

Link to original post