Trending Now

Marketing surveys are wrong

Yeah, I know it is a sensationalist headline. Sue me. I am reacting to everyone constantly telling me that social media listening provides “wrong” information. They do. Every technique that purports to provide insights into markets has some kind of error in them because this isn’t an exact science. But to pooh-pooh social media listening on the grounds of inaccuracy is just as dumb as my headline that takes to task all market research. What’s really true is that different research tools have different strengths and weaknesses and we should understand them so that we use the right ones for the right purposes.

So let’s see where social media listening falls short. It is certainly true that if you take the average listening tool and look at all the conversations that it marks positive or negative that some of them will be wrong. [Full disclosure: I am the Chief Strategist for Converseon, which makes a way-better-than-average listening tool.] Now, understand, if you give those same conversations to human beings to analyze, they won’t always agree with each other, so it must be that some of them are wrong, too. But no matter, let’s agree that some of the conversations marked by sentiment analysis tools are wrong.

Forecasts by polling and market research compa...
Photo credit: Wikipedia

Some of the answers in market research surveys are wrong, too. No, I don’t mean that SurveyMonkey screwed up the answer. I mean that people gave the wrong answer. They might have said what they thought they would do when they would actually do something else in the real situation. Or they might have told you something nice instead of the unvarnished truth.

Now, I know that everyone knows this already and we just accept the fact that some of the answers are wrong. So, why can’t we accept that for social media listening, too? Understand that I am making no excuses for the technology. We should keep working on it. But it is worth something to know the answer even if it isn’t completely accurate.

Here’s why. Surveys aren’t just wrong–they are incomplete. We only get the answers to questions we know to ask. So there might be serious insights that we miss because we don’t think to ask. Social media conversation is unprompted, so that doesn’t happen there.

But isn’t social media listening also wrong because we can’t isolate all the relevant conversation? I mean, entering keywords for Boolean queries can’t isolate the right conversation  Sprint can’t separate the conversations about their cell phone service against sprint car races or lots of other meaningless stuff–meaningless to them, anyway. But advanced social media listening tools can separate the wheat from the chaff through machine learning.

And wouldn’t you expect to have to do that? I mean, do you think you have the answer to your market research questions when SurveyMonkey hits 1000 responses? No, you don’t. You need to ensure those responses are from the right people–that the responses are properly weighted to reflect your market. If you don’t do that, the insights are wrong. So shouldn’t you give social media that same break?

Let’s not react against every new thing because it has problems. The old things have problems, too. Old problems aren’t less important just because we are accustomed to them.

Enhanced by Zemanta

Mike Moran

Mike Moran is a Converseon, an AI powered consumer intelligence technology and consulting firm. He is also a senior strategist for SoloSegment, a marketing automation software solutions and services firm. Mike also served as a member of the Board of Directors of SEMPO. Mike spent 30 years at IBM, rising to Distinguished Engineer, an executive-level technical position. Mike held various roles in his IBM career, including eight years at IBM’s customer-facing website,, most recently as the Manager of Web Experience, where he led 65 information architects, web designers, webmasters, programmers, and technical architects around the world. Mike's newest book is Outside-In Marketing with world-renowned author James Mathewson. He is co-author of the best-selling Search Engine Marketing, Inc. (with fellow search marketing expert Bill Hunt), now in its Third Edition. Mike is also the author of the acclaimed internet marketing book, Do It Wrong Quickly: How the Web Changes the Old Marketing Rules, named one of best business books of 2007 by the Miami Herald. Mike founded and writes for Biznology® and writes regularly for other blogs. In addition to Mike’s broad technical background, he holds an Advanced Certificate in Market Management Practice from the Royal UK Charter Institute of Marketing and is a Visiting Lecturer at the University of Virginia’s Darden School of Business. He also teaches at Rutgers Business School. He was a Senior Fellow at the Society for New Communications Research and is now a Senior Fellow of The Conference Board. A Certified Speaking Professional, Mike regularly makes speaking appearances. Mike’s previous appearances include keynote speaking appearances worldwide

Join the Discussion

Your email address will not be published. Required fields are marked *

Back to top Back to top