Big Data and Day Trading: The Good, the Bad, the Ugly.

Read more and subscribe at georgejmount.com 

These days, I can’t check my LinkedIn without seeing something about data science. Everybody’s on CodeAcademy and Coursera, dreaming of that sexiest of job titles: “Data scientist.”

Read more and subscribe at georgejmount.com 

These days, I can’t check my LinkedIn without seeing something about data science. Everybody’s on CodeAcademy and Coursera, dreaming of that sexiest of job titles: “Data scientist.”

I’m all for making data an asset, but I see some troubling comparisons between big data and the day trading of the 1990s.

While I was only in grade school back then, I studied it a bit in college and grad school.

I make no bones about being either a stock trader or a data scientist. But I do have a master’s in finance and a post-grad certificate in business analytics from two leading business schools, so I hope my comparisons aren’t completely unfounded.

If you are either of these, please share your thoughts in the comments. And go easy on me, I was preparing ribs for the holiday weekend while writing this post

I see some similarities between data science and the day trading of a generation ago. They share similar origin stories and fatal flaws.

Here’s how they’re similar — the good, the bad, and the ugly.

The good: democratized information.

Both day trading and big data are a result of the opening up of information to the average end-user.

Back in the 90s, for the first time, traders could easily check prices during the day, run models, and trade from anywhere on the planet.

Similarly today and big data. Using cloud servers and cheap BI software, anyone can crunch huge amounts of data at very little cost.

Democratization and low barrier entries are great. The economy has benefitted tremendously from the abundance of capital and information that these two trends have brought about. 

I am all for this part. Sadly, other trends aren’t so good.

The bad: get rich quick.

I know people who dream of taking Coursera classes, installing a home server, and striking it big as a data scientist. 

This reminds me of the disgruntled 90s employer who would be crushing it in his slippers, if he would only quit his job.

We saw what happened to most day traders. They simply weren’t as smart as they thought. Some did crush it, but it’s nearly impossible to keep out-guessing the market.

I think the same is going to happen in data science. It’s not a 1:1 analogy because some datasets are proprietary and not publicly traded. But it is happening, as more and more data is becoming crowdsourced.

The other setback is hubris and theory, which brings me to my next point. 

The ugly: we’re too smart for theory

The backseat treatment of theory is the most devastating aspect I that see in both the day trading and big data phenomenon.

There’s an attitude that we have mastered the data so thoroughly that we don’t need to consider human behavior or classical economic theory. 

I think of Long Term Capital Management or even the housing bubble. So sure were they that their financial engineering could withstand any calamity, they overlooked catastrophes that simple economic theory could have helped predict.

I wonder if there’s an element to this with big data. It has increasingly become “data mining.” 

While classical statistics may have asked first and shot later, data mining shoots first and ask questions later. Theory takes the back seat. 

This has its place. But it is easy to become overconfident in models that are developed this way. You see that the model works, but you’re not quite sure why. This is dangerous. This is partly where day trading went wrong. And this is where data science veers, as well.

Before you place a put on my stock…

What do you think of these comparisons? Unfounded, partially true, or should I just stick to drinking beer this weekend? 

George J. Mount holds a bachelor of arts magna cum laude in economics from Hillsdale College, a master of science in finance from Case Western Reserve University, and a postgraduate certificate in business analytics from Indiana University. He blogs regularly atgeorgejmount.com.