Text Analytics WIIFM (What’s in it for Me?)

Every day or so, someone asks me what’s hot in analytics now. My answer is simple – text. Text analytics, the practice of converting text to data, is today’s data analysis hotbed, and will be for the next decade and beyond.

Text analytics is no walk in the park. Even humans find it difficult to clearly determine the meaning and implications of every bit of text they view, and this is a task that machines don’t do as well as humans. The current state of the art is far from perfect, as the best practitioners and vendors will surely admit. So why is the market growing 25% per year, now reaching $835 million annually? That’s not curiosity spending. Decision makers are shelling out the money because there is something in it for them.

The question is: what’s in it for you?

What are those decision makers getting that motivates them to open the checkbooks and spend? What does text analytics do for them? What do they know and how can you learn it for yourself?

Literature from the vendors who are enjoying revenue from the text analytics boom is remarkably vague about how customers can improve the bottom line with language technology. And most presentations and articles on the topic are academic; they focus on the nitty-gritty details of creating and refining the technology, but say nothing about its value for business. There are no step-by-step guides to profiting from text analytics.

One excellent resource that everyone interested in practical uses for text analytics should read is Altaplana’s “Text/Content Analytics 2011: User Perspectives on Solutions and Providers.” The study’s author, Seth Grimes, is an expert on the text analytics market (as far as I know, he is the only analyst making a career focusing on text analytics), and the report provides details on how text analytics is being used and the drivers behind its use. It even includes a section on how organizations are measuring return on investment (ROI).

And what are text analytics adopters reporting about ROI? Not what you’d hope. The majority of participants in the study are not yet reporting positive returns.

Does that mean text analysis is a waste of money? No. Still, it’s clear that there is nothing magic about linguistic technology. Invest without a good plan and you’re not likely to get positive returns.

Messaging for text analysis products and services is long on concepts that sound good – listening to the voice of the customer, gaining insight, unifying views – yet short on assessing the business value of these things. Exactly how much is an insight worth? My hat is off to the sales teams who bring in millions selling with messaging like that. I’ve been promoting the use of text analytics for a dozen years or so, and have always had a lot more success where there was a clearly defined opportunity for cost savings or increased revenue generation.

The experience of the respondents in the Altaplana study is disturbing, not merely because so many of them have not achieved positive returns on investment, but also because positive returns are a realistic and reachable goal. Those seeking to adopt text analytics should take note, and begin with the basics of identifying real business problems, assessing the associated costs, setting goals and carefully examining the potential to reach those goals with the aid of text analytics. At the same time, vendors must refocus on applications that support clearly measurable cost reductions or revenue increases.

Consider just a few examples of text analytics applications and how they can impact costs and revenue:

Market Research – survey researchers replace hand-coding of open ended survey responses with automated coding. There is well-defined cost savings – no more expense for hand-coding. Reduced turnaround time may translate to greater revenue opportunity. Automated coding is also a more consistent process than hand coding.

Pharma – text-analytics enhanced literature research enables researchers to identify useful information more quickly. Productivity of valuable researchers is improved, with significant revenue benefits associated with faster discovery and drug development.

Retail – enhanced search capability keeps customers on the retailer’s site and increases conversion rate and total sales – straightforward revenue increases.

These examples integrate the same ideas widely used to promote text analytics today. After all, the point of a survey is to hear the customer’s voice. All research is intended to glean insight. But they are different in that they speak directly to how those benefits translate to changing the bottom line. The real stories are far more complex than the few words written here, but if you can’t write a statement of a few words explaining how the application will make or save money, you certainly haven’t thought it through.

Text analytics, what’s in it for you? Could be millions, but you won’t get there just by investing. You must ask hard questions and plan a process that connects the dots from insight to concrete revenue gains and cost savings.