What does it take to work with big data? Surprisingly, a lot. Here’s what you need to know to get into this exciting career.
The Skillset
What does it take to work with big data? Surprisingly, a lot. Here’s what you need to know to get into this exciting career.
The Skillset
There are a lot of skills you need to have, just to be a competent IT professional. To start off, Apache Hadoop. It’s entering its second decade now, but there’s still a lot of room for improvement in architecture and development. And, Hadoop can be a fussy thing to work with, requires care and feeding by proficient techs.
The Apache Spark certification at Simplilearn is another skillset worth investing in. If Hadoop is a known quantity in big data, Spark is the black horse. Because it’s still young, there’s plenty of room to grow in this industry, and it requires technical expertise to program and run.
NoSQL is on the operational side of the big data house. Databases like Couchbase and MongoDB are taking over jobs that used to be done by SQL databases like Oracle and IBM DB2.
NoSQL is often the database of choice in mobile apps, and that data is sent through Hadoop – meaning having this training and skillset will be a nice addition to your Hadoop training.
Machine learning is the next frontier. Imagine computers that (practically) think for themselves. One of the hottest fields in big data is data mining and machine learning. Predictive analysis and personalization are in high demand now, and the trend is only increasing.
Statistical analysis. This is what big data is, really. If you’ve got a background in quantitative analysis and mathematics (statistics), you’re halfway there. Add in expertise in R, SAS, Matlab, SPSS, or Stata, and you’re in for a very bright future.
SQL is the data-centric language that’s more than 40 years old. But, the workhorse still has a lot of life in it, even in the age of big data. While it certainly won’t be used for all big data challenges, it can be used in Structured Query Language.
Data visualization is another skill you’ll want to look into. big data can be difficult to understand. There’s just so much of it. Think of how an average person would take in a seemingly endless string of numbers and data points. This is where visualization comes in. When you can present a visual of what’s happening with the data, it makes things much easier to “get.”
If you want to geek out on the raw numbers, fine. If you want to communicate those numbers to other people, and translate the raw data into dollars and cents, you need visualization.
Finally, you’ll need a lot of general purpose programming language skills. Knowing Java, C, Python, or Scala will help you eek out an edge over others whose skillsets are confined to just analytics.
Having A Curious Mind Helps
While degrees, advanced skillsets, and great academic achievement helps, it’s nothing compared to having a curious mind and the ability to think outside the box. Sometimes, the hardest part isn’t coming up with the data, or even organizing it. it’s communicating its benefit to others who don’t understand. And, if you can’t do that, you don’t have much of a career. You also have to know what you want to do with all that data, and that’s where being creative comes in.
The problem with big data is that there’s so much of it, it almost seems overwhelming. In many ways, it is overwhelming. But, creativity in application and usage will help you throw away what’s unnecessary while focusing intensely on the important stuff.
A Scientific Mind Also Helps
Never giving up, a mind devoted to truth, testing, and analysis. This is what’s required in the big data industry. Professionals aren’t just curious and educated. They’re also dedicated – dedicated to discovering the truth behind the data. They’re not afraid to be wrong. They’re not afraid to do research and have it turn up nothing. This is the kind of bravery you need to survive in this industry.
Communication Skills Required
Communication is probably the most underrated skill in IT. Most professionals either give lip service to it, or are completely oblivious to the need for better communication. To some extent, there’s a personality that’s well-suited to IT – an introverted, highly intelligent and capable person who likes to work alone and doesn’t want to socialize with others.
But, in today’s connected world, that personality type has to break out of his or her shell and interact with customers. Being able to communicate complex ideas in simple-to-understand language is a must. Customers don’t understand technical jargon and, although it’s required for the professional, the client simply wants the solution to work without a lot of fuss.
When you’re speaking with non-IT professionals, the key is to communicate in a language they’re likely to understand. Ebb and flow between high level concepts and concrete examples.
