Yeah, I know it is a sensationalist headline. Sue me. I am reacting to everyone constantly telling me that social media listening provides “wrong” information. They do. Every technique that purports to provide insights into markets has some kind of error in them because this isn’t an exact science. But to pooh-pooh social media listening on the grounds of inaccuracy is just as dumb as my headline that takes to task all market research. What’s really true is that different research tools have different strengths and weaknesses and we should understand them so that we use the right ones for the right purposes.
So let’s see where social media listening falls short. It is certainly true that if you take the average listening tool and look at all the conversations that it marks positive or negative that some of them will be wrong. [Full disclosure: I am the Chief Strategist for Converseon, which makes a way-better-than-average listening tool.] Now, understand, if you give those same conversations to human beings to analyze, they won’t always agree with each other, so it must be that some of them are wrong, too. But no matter, let’s agree that some of the conversations marked by sentiment analysis tools are wrong.
Some of the answers in market research surveys are wrong, too. No, I don’t mean that SurveyMonkey screwed up the answer. I mean that people gave the wrong answer. They might have said what they thought they would do when they would actually do something else in the real situation. Or they might have told you something nice instead of the unvarnished truth.
Now, I know that everyone knows this already and we just accept the fact that some of the answers are wrong. So, why can’t we accept that for social media listening, too? Understand that I am making no excuses for the technology. We should keep working on it. But it is worth something to know the answer even if it isn’t completely accurate.
Here’s why. Surveys aren’t just wrong–they are incomplete. We only get the answers to questions we know to ask. So there might be serious insights that we miss because we don’t think to ask. Social media conversation is unprompted, so that doesn’t happen there.
But isn’t social media listening also wrong because we can’t isolate all the relevant conversation? I mean, entering keywords for Boolean queries can’t isolate the right conversation Sprint can’t separate the conversations about their cell phone service against sprint car races or lots of other meaningless stuff–meaningless to them, anyway. But advanced social media listening tools can separate the wheat from the chaff through machine learning.
And wouldn’t you expect to have to do that? I mean, do you think you have the answer to your market research questions when SurveyMonkey hits 1000 responses? No, you don’t. You need to ensure those responses are from the right people–that the responses are properly weighted to reflect your market. If you don’t do that, the insights are wrong. So shouldn’t you give social media that same break?
Let’s not react against every new thing because it has problems. The old things have problems, too. Old problems aren’t less important just because we are accustomed to them.
