Donald Trump’s presidential victory stunned experts from both sides of the aisle. Polls across every battleground state showed Hillary Clinton with solid leads, yet few of those polls seemed accurate on election night. State after state reported voting results at odds with what almost every poll showed. What happened?
We’ll probably be asking that question for years and years. Pollsters are asking themselves, “What did we miss?”
I don’t know which of the network news channels I was watching when someone said, “Someone told me that there were a lot of ‘leaners.'” When questioned, he aped someone leaning in and whispering, “I am going to vote for Trump.” His point was that at least some of the people who voted for Trump did not necessarily want to be open about it, and perhaps did not let on with the pollsters.
Another story I heard was that the only poll that seemed to predict a Trump victory was an unusual poll from the University of Southern California and the Los Angeles Times that was a time series over months where they asked the same 3,000 people over and over throughout the campaign who they were supporting. It’s possible that those voters built up a relational rapport over time and perhaps answered more honestly than those who were confronted with one pollster once.
In addition, they asked what is the percentage that you are “for” the candidate, so they could detect the difference between someone who was 100% in favor of Clinton last week and is 60% in favor this week—when the other polls just would ask, “If the election were today, who would you vote for?” which misses that nuance.
Also, affecting all polls, it’s been widely reported that many Trump voters have identified themselves as being angry. I don’t know if this is true, but when I am angry, I don’t always know what I am going to do. I might not know what I am going to do until the moment that I have to do it. It’s possible that asking people days and weeks beforehand how they will vote is asking them what they honestly did not know at the time.
So what does this have to do with market research? Plenty.
Electoral polls are actually one of the purest forms of market research. And, we can judge the polls’ accuracy by the outcome on election day. Businesses conduct market research all the time, but have no election day results to prove that it works. Could your market research be equally flawed because your customers don’t want to tell you the truth? Or because your questions fail to capture the nuances of their thinking?
One of the problems with traditional market research is that it is usually based on survey questions. How those questions are formulated matter. Human beings don’t always want to give you their unvarnished thinking. People sometimes don’t know what they will do in the future.
I am not smart enough to know how to fix election polls so that they are more predictive than this year’s have been. But market research is not as hard as election research, because we can watch people making decisions every day, such as what content to share with others and actual purchasing decisions. We can also listen to people talking about these issues unprompted in social media, which can help remove some of the question bias.
Introducing these new sources of data also introduces new biases. How do you know you have the right sample of people? How do you know that you are sampling the right social conversation? Are those that are vocal more negative than those that are positive?
But adding new data sources with new biases should add to our picture of the market by supplementing our old data with its known biases. We might not have the accurate answer every time that we all want, but we might at least eliminate some blind spots. It’s the blind spots that led to polls universally predicting the first woman president in the US, not the first president without political and military experience. Do you have some blind spots in your market research? What can you do to eliminate some of them?
