The Truth About Data Scientists – It’s A Lot Bigger than Big Data

With the  explosion of big data and analytics in popular corporate culture, Data Scientists are suddenly the cool kids at the lunch table. I love that the career I’ve dedicated my life to is getting such traction, but I’d like to clear up a few things.

While you may have only just heard of the term, Data Science isn’t new. I’ve spent 20 years working with multiple Fortune 100 and Fortune 500 companies as a data scientist – helping them clear up the data and bring logic to an emotional world.

What exactly is a Data Scientist?

A Data Scientist is tasked with reviewing the information that’s collected. The data can take up terabytes of hard drive space, and thousands of pages in excel. It’s just not feasible for a single person to manually sort, compile and draw logical conclusions from that much information.

So, we turn to the science of data – and more recently, the assistance of AI. We create and deploy algorithms that allow us to efficiently sort large chunks of data. We’ll sort information based on schemas that help us identify relationships within databases, and better identify relationships within metadata.

The information that’s collected is incredibly complex with multiple variables. Even a simple collection of yes/no (binary) answers to a survey need to be compiled within the context of a broader collection of data. This involves referencing multiple data sets to deliver answers that are logically sound and stand the test of time.

No, Big Data and Data Science aren’t the same thing!

It’s easy to make the mistake of thinking that Big Data and Data Science mean the same thing. Big Data is today’s buzzword, and it literally means the collection of huge amount of information based on a series of interactions or events.

Data Scientists analyze Big Data with the help of multi-variable calculus models. This is different from traditional computer coding. The equations and calculations we perform are not designed to create software or websites. Instead, we complete exhaustive analysis that delivers critical information to decision-makers within the c-suite.

The first experiences of a newly minted Data Scientist…

After graduating college with a mathematics degree and some coding experience, you’ll be shocked at how different the real-world is from the college experience. Textbook analysis involves a logical world.

The world is not always logical. Human emotion and sentiment adds a complex layer to data analysis that requires greater context to the information that is collected. If you enter the field expecting to write a little code, run a program and deliver the results to a thankful audience, buckle-up, you’re in for a shock.

People don’t behave the way college textbooks tell you that they will. Get used to surprising information and unpopular analysis.

Companies certainly appreciate information about how long someone stays on a website. But decision-makers demand to know how well their products are selling, and what factors are impacting their performance in the marketplace.

It’s our job to take the stats and apply every tool in the toolbox to create a complete picture. You’ll get pushback from people with limited information. For example, the website guys are going to tell you everything is great because site bounce rate is down. But the CFO is going to say that something is wrong because profitability is down. You can’t make everyone happy – and that’s not your job.

It’s your job to translate the real-world into corporate ROI.

You’ll analyze behavior trends, adoption / ROI, along with exhaustive mathematical analysis (data mining, optimizations, predictive analytics, etc.) to help understand the past and predict the future. Big Data is just one source of information. We look at the entire world, and direct further data collection based on the way we see data coming together.

If predicting the future sounds exciting to you, you’re in the right field. Especially when you consider all of the advances in machine learning and database management. This is an exciting field that requires you to remain on your toes, constantly learning about all of the things that impact a final determination or solution.

Do you spend your day challenging assertions and playing “What if?”. Learn to code, dive into linear algebra and get experience in the corporate world. All of the things you learn will combine to help you provide contextual advice that improves a company’s ability to turn information into profit.