Is Big Data Good or Evil?

With all of the lawsuits working through the courts and all of the scary possibilities being discussed in the media, it has led some people to assume that big data is inherently evil. Once you believe that big data is evil, a natural response is to try and shut down the collection and analysis of big data to the maximum extent possible. While big data certainly has risks, it would be a classic case of throwing out the baby with the bathwater if the use of big data is shut down.

I had a meeting with a group of state legislators and executives this past week to discuss how analytics and big data could help their state better control costs, limit fraud, and more efficiently deliver state services through the use of big data, more sophisticated analytics, and updated data management platforms. The issues of privacy and the misuse of data were raised, which led to some intense discussion. At the heart of the discussion was whether or not having a state collect massive amounts of data is a good idea due to the risk of misuse and abuse by those in power.

This is a legitimate concern and one that can be supported by a variety of governmental abuses in recent times. Just look at investigations into the NSA and the IRS within the United States government for examples of the type abuse that is of concern as more data is collected. While I am certainly against any and all abuses of data, I also think it is important to avoid overreacting. The correct response is to stop the abuses, not to stop any activity that could lead to abuse.

To answer my own question, I don’t think that big data is inherently evil. Instead, big data is simply neutral information that has the potential to be used for good or evil depending on who is given access and what their intentions are. Because of this, I believe that focus should be placed on aggressively prosecuting and stopping any abuses alongside aggressively enabling and pursuing the good outcomes that can come from big data. Money is similar to data in this way. Money helps all sorts of wonderful things happen around the world. At the same time, money is used to commit acts of evil. Most people don’t view money as evil, however, and don’t suggest that we do away with money to stop the abuses money is a part of.

As I spoke with the state officials, I pointed out examples of how robust data can provide immense benefits. Consider state social workers assigned to monitor families with a history of child abuse. When data on past interactions with the family isn’t available to a new case worker, children can end up needlessly injured or even dead. In my home state of Georgia, for example, there has been a lot of attention on children that died because case workers didn’t have critical information on the known risks the children were facing.

One state official pointed to a valid concern related the highly sensitive information collected by these state workers. Making information easily available means that low paid, non-technical workers will have at their fingertips highly private, sensitive information. That information clearly can be abused and so I fully agree that the risk for abuse is present. However, I believe that the cases where such information will save a child can far surpass the cases of abuse. This will be especially true if those with access to the data know that misuse will not only place their job on the line, but that they will be aggressively prosecuted if they breach the trust they have been given.

Every time we drive in a car, we are risking our lives. At any moment, someone else can slam into us and kill us even though we did nothing wrong. It is tragic when this occurs, but it occurs rarely enough that we consider it an acceptable risk. The many benefits we gain from having the ability and freedom to move about as we desire more than make up for the risks. Nobody seriously suggests banning cars from the road in order to stop the multitude of fully avoidable automobile deaths that occur each year. As a society, we have determined that the benefits are too great compared to the risks.

My belief is that we need to follow the same approach with big data and analytics. The benefits of using big data properly can be immense. No matter how careful we are, however, that same data will at times be misused. Our goal should be to minimize the occurrence of abuse and make the penalties significant enough to dissuade most people from even trying. If state governments, as well as other organizations, are able to leverage big data in only a fraction of the positive ways that it can be leveraged, we’ll all be better off for it.