The Top Challenges with Sentiment Analysis

Did you know that there are more than 200 tools and platforms that can help you track and assess how many times your business or brand has been mentioned in social media channels?

Did you know that there are more than 200 tools and platforms that can help you track and assess how many times your business or brand has been mentioned in social media channels?

And many of these listening platforms do more than just the basic monitoring. In fact, they now offer integrated approaches to get the right information to the right parts of your business: product development; customer support; public outreach; lead generation; market research; and campaign measurement.

It’s a big responsibility and commitment to listen to your customers. That’s why businesses are making it a priority to enhance their social monitoring efforts and sentiment analytics to pick up and decipher what their customers are saying about them.

Analytics strategist, Seth Grimes (@SethGrimes) says sentiment analysis lets marketers et al., “get at root causes, at explanations of behaviors that are captured in transaction and tracking records.” Sentiment analysis lets you better target your marketing, detect opportunities and threats faster, protect the reputation of your brand, and most importantly, turn a profit.

Still, it’s important for data scientists to use caution when accepting customer statements at face value since context has such a great bearing on meaning. Analyzing natural language is difficult enough. Sarcasm or other forms of derisive language are extremely problematic for technologies to interpret.

For instance, let’s say Karen learns from a Facebook friend that an electronics company has just started charging customers a support fee for a popular product that had historically been free. Karen posts the following response on Facebook: “Oh, that’s just great.”

Taken literally, or by narrowing the analysis to positive or negative words that are made about the electronics company in social media, Karen’s statement would be interpreted to mean that she’s pleased by the change in the support policy. But more than likely, she’s simply being sarcastic.

In many cases, analytics teams are evaluating larger samples of customer statements to help spot potential product issues or indicators that could signal customer churn.

They also do this to help identify possible trends within different customer segments. It’s not cost effective nor efficient for data scientists to analyze the sentiments of individual customers, with the possible exception of companies that market a limit number of high-end products such as luxury shoes.

This also brings up the issue of scale. One of the biggest issues with analyzing tweets made on Twitter, based on research of the service, is that only about 60% of the people who use the service are actually tweeting. Forty percent of Twitter users don’t tweet or haven’t tweeted in 30 days. That means that more than one-third of the people on Twitter are simply observers.

Taking this a step further, let’s say an automotive maker uses Twitter to analyze comments that are made about its competitors. Taken on its own, such an analysis will capture the opinions of a subset of Twitter users. But it’s not necessarily a fair representation of the universe of Twitter users.

Sentiment analysis tools continue to evolve and will continue to improve over time. In the end, organizations that augment sentiment analysis with analysts who are able to interpret context in comments and take comprehensive approaches to sentiment analysis are those that are likely to benefit most.