Big Data Holds Big Promise for Government

Much of the focus around using business analytics to interpret big data has revolved around the private sector, as companies scramble to add new customers, identify the most profitable existing customers and bolster revenue by culling through the reams of unstructured data from social networks, mobile devices and other sources flowing into their networks.

But government also stands to benefit from combining traditional transactional data with new data sources to advance urban planning and city management, stop fraud and strengthen services.

Scientists in Singapore, for example, are developing software to turn cities into “real-time control systems” that combines all sorts of data feeds like information about rainfall and the location of taxis so the government can match the demand and the supply of taxis in specific weather conditions, particularly when it rains – a common occurrence in Singapore.

Furthermore, the Centre for Advanced Spatial Analysis (CASA) at University College London is combining data from London’s Oyster cards – used to pay for public transport – and Twitter messages. Tube-travel patterns are regular: people who enter the system at one station tend to leave it at a particular other one. Twitter messages reveal a city’s structure and its activity.

Some academics like Michael Batty, the director of CASA, see real benefits from synthesizing patterns and regularities in big data into a “science of the city,” much like physics or biology.

The British government could save up to £33bn ($51 billion) a year by applying big data analytics to departmental information, according to a new report from the think tank the Policy Exchange.

The Policy Exchange estimated up to £22bn ($32 billion) could be found by running the departments more efficiently; up to £3bn ($4.7 billion) by reducing fraud and error; and £8bn ($12 billion) by collecting a greater share of unpaid taxes.

Big data has the potential to transform the work of government agencies, unlocking advancements in efficiency, the speed and accuracy of decisions and the capability to forecast, according to a separate report from MeriTalk.

The amount of data that the government collects is growing: 87% of government IT professionals report that their stored data grew in the last two years; and 96% expect their data to grow in the next two years (by an average of 64%), according to the report. Unstructured data makes up 31% of data held by the government, and the percentage is on the rise.

While government agencies collect massive amounts of data, the report finds that only 60% of IT professionals say their agencies analyze the data collected, and less than 40% say their agencies use the data to make strategic decisions. While 60% of civilian agencies are exploring how big data could be brought to bear on their work, only 42% of defense and intelligence agencies are doing the same.

In an effort to help government agencies harness the power of big data, the Obama Administration has recently announced a “Big Data Research and Development Initiative” that promises more than $200 million in new research and development investments in big data.

The White House has created the initiative to:

Advance state-of-the-art core technologies needed to collect, store, preserve, manage, analyze and share huge quantities of data
Harness these technologies to accelerate the pace of discovery in science and engineering, strengthen national security and transform teaching and learning
Expand the workforce needed to develop and use big data technologies

Next steps:

Subscribe to our blog to stay up to date on the latest insights and trends in big data and data analytics.
Check out our complimentary “5-Minute Guide to Business Analytics” to find out how user-driven “analytic” or “data discovery” technologies help business and technology users more quickly uncover insights and speed action.