…spontaneous complaints and compliments are to customer loyalty management. Like these forms of customer experience feedback, tweets are unsystematic, unorganized, and representative of who knows what underlying sentiments in the broader universe of individual experiences.
Imagine for a moment that we could extract from the Twitterverse all of the tweets about about a particular brand. A little text analysis might help us categorize the tweets along some dimension, such as positive versus negative, and we might dig deeper for specific key words. How many times do the key positioning statements for the brand turn up in these tweets, for example?
Even with an exhaustive set of tweets about our brand (within some time frame, of course) we’ll have only a sample of all the existing conversations about the brand. More important, we don’t know exactly what that sample represents. Most certainly it is not a random sample of all conversations about the brand.
Spontaneous or casually solicited complaints and commendations – long a staple of customer experience management – are a lot like the spontaneous brand conversations we might find in the Twitterverse. There may be substantive content, but we know next to nothing about potential sampling error and selection bias. In the case of the tweets we have both a population restriction (only some customers will use Twitter) and a potential selection bias (only some Twitterers choose to tweet about the brand). One of the main arguments for probability sampling of customer experiences (via systematic customer satisfaction measurement programs) is the potential bias in spontaneous customer feedback.
One of my favorite social science metaphors comes from Unobtrusive Measures: Nonreactive Research in the Social Sciences (Webb, et. al., 1966). “Outcroppings,” a concept from geology, are “those points where theoretical predictions and available instrumentation meet.” Think of an exposed seam of coal in a particular geologic formation. A single outcropping doesn’t tell us much about the way in which coal seams formed or the best places to look for coal. More outcroppings lead to better understanding and prediction.
Tweets and other consumer-generated social media content are best used in combination with other outcroppings – systematic research (experiments and surveys of probability samples of consumers) and other behavioral observations (such as transactional data).
One of the unintended (but positive) consequences of the migration to Internet-based survey research has been a critical examination of some fundamental assumptions about survey research. How often did you hear concerns raised about non-response, non-coverage, and non-representative samples when the primary method of survey sampling was random digit dialing. We were lulled into what seems to be a false sense of security by a well-defined sampling frame and procedures that insured that we had a probability sample of telephone numbers.
Of course, survey research always has been only one of many possible outcroppings in the quest for customer knowledge. I’m encouraged that the shortcomings of online survey research and other data created through online behavior has sparked serious debate about how we know what we know in the realm of customer knowledge.
Copyright 2009 by David G. Bakken. All rights reserved.
