Barack Obama may have appointed the US Government’s first ever Chief Data Scientist – DJ Patil, formerly of LinkedIn, eBay and Skype – but the administration has been committed to the use of Big Data for some time. From pledging to put all government records into the public domain, to less commendable initiatives such as mass interception of telecoms data, it is clear that the powers-that-be of the world’s most powerful nation are far from ignorant of the importance of the Big Data revolution. Last year a group of White House officials charged specifically by the president to examine the current and potential uses of Big Data concluded that it presented both great opportunity, and great danger. The report stated: “While big data unquestionably increases the potential of government power to accrue unchecked, it also hold within it solutions that can enhance accountability, privacy, and the rights of citizens. Properly implemented, big data will become an historic driver of progress, helping our nation perpetuate the civic and economic dynamism that has long been its hallmark.” The report contained stark warnings about the impact that Big Data could have on human rights issues, such as use of predictive analysis to deny healthcare or education to citizens. But overall it concludes that “Positive benefits of technology are (or can be) greater than any new harms.” Certainly uses are already being found for Big Data in fields such as heathcare, security and education which can potentially have huge positive impact. These include:
- The CIA helped to fund Palantir Technologies, which produces analytical software designed to stamp out terrorism (by among other things predicting the location of roadside bombs) and crack down on cyber fraud, by identifying transactions which follow patterns commonly displayed during fraudulent activity.
- Law enforcement agencies (at federal, state and county levels) have access to sophisticated ALPR (automated licence plate recognition) software which alerts them to anyone in the vicinity with an outstanding warrant. And predictive technologies are used by several police departments to predict flashpoints where crimes may occur, as well as link particular crimes to particular repeat offenders.
- The Department of Transportation also uses licence plate recognition as well as cameras to monitor the flow of people as they travel by plane, train and automobile, generating insights as to where infrastructure investment is necessary, as well as predictions about how we are likely to change the way we travel in the future.
- In education, thanks to the huge increase in the amount of learning activity carried out online (both in traditional school environments and through distance learning), massive amounts of data about the way we study and learn is becoming available.
- The US Department of Agriculture of supports the agricultural economy through research and development of new technologies, many of which are Big Data-driven. One recent breakthrough is increasing the yield of dairy herds by identifying bulls most likely to breed high-yielding cows from their genetic record.
- Government agencies involved in healthcare, such as the Centers for Disease Control track the spread of illness using social media (you can see the FluView results reported here in realtime), while the National Institutes of Health launched an initiative called Big Data to Knowledge (BD2K) in 2012, to encourage innovation based on data-derived insights. One project funded by the Government even aims to spot the early signs of suicidal thinking among war veterans, based on their social media behavior.
Obama himself has been called the Big Data President by the Washington Post – responsible for launching data.gov, and a £200 million investment in Big Data research and development. He also pledged to make public all data that the government collects which is not private or restricted for national security reasons. Indeed privacy concerns are often seen as the biggest challenge presented by the rise of Big Data, and the US Government has identified several potential areas that could be infringed on through inappropriate use of data, including rights to secrecy of personal information and rights to remain anonymous while exercising free speech. The National Security Agency (NSA) “warrantless wiretapping” controversy is probably the best (by which I mean worst!) example of this line clearly being overstepped, where it emerged that the agency was collating data on the private conversations of millions of people in the US and overseas. The excuse was given – by the Bush administration and then repeated by the current administration – that the snooping was necessary for reasons of national security. However many still hold that it appears to be illegal under the constitution which guarantees freedom from intrusion by the state into private lives. Big Data use in Government certainly presents big challenges – officials and politicians have a fine line to tread if they do not want to come across as attempting to implement a real-life version of Orwell’s Big Brother. It is certainly terrifying to think of the uses a modern-day Hitler or Stalin could find for the data and technology we have available today. After all, if the US Government can use it then so can any ruling administration – many of which are subject to even less regulation, and their citizens less free to scrutinize and hold them to account. However with the right balances in place – such as robust regulation and protection of “whistle blowers” – I believe it can be used for great positive social change – as demonstrated by the projects I’ve mentioned in this article. But whether Obama will go down in history as the Big Data President or Big Brother President remains to be seen. What are your views on the use of big data in government? Please share them in the comments below.