Big Data: Good or Evil?

An article in the New York Times stokes fears of Big Data:

Social scientists are trying to mine the vast resources of the Internet — Web searches and Twitter messages, Facebook and blog posts, the digital location trails generated by billions of cellphones — to do the same thing.

An article in the New York Times stokes fears of Big Data:

Social scientists are trying to mine the vast resources of the Internet — Web searches and Twitter messages, Facebook and blog posts, the digital location trails generated by billions of cellphones — to do the same thing.

The most optimistic researchers believe that these storehouses of “big data” will for the first time reveal sociological laws of human behavior — enabling them to predict political crises, revolutions and other forms of social and economic instability, just as physicists and chemists can predict natural phenomena.

The article then goes on to cite fears of a society run amok: “I have Total Information Awareness flashbacks when things like this happen”. Those fears may well be justified; but “Big Data” isn’t where the finger of blame should be pointed. Certainly, Big Data enables applications that simply aren’t possible with the small samples of traditional statistical methods: 

“Big data allows one to move beyond inference and statistical significance and move toward meaningful and accurate analyses,” said Norman Nie, a political scientist who was a pioneering developer of statistical tools for social scientists and who recently formed a new company, Revolution Analytics, to develop software for the analysis of immense data sets.

Applications of Big Data can of course be nefarious (like the Total Information Awareness project), but they can be beneficial too: tools like Google Translate and the iPhone’s new Siri voice automation system simply wouldn’t be possible without an enormous corpus of data to match against. But neither case makes Big Data inherently good or evil: data is just data. It’s how it’s used that counts.

New York Times: Government Aims to Build a ‘Data Eye in the Sky’