Much has been written about, discussed, and broadcast about the apparent failure of public opinion and political polling in 2017.
To be clear, there was not an issue with the data in the U.S. election; almost all reputable polls were within the margin of error nationally. There were certainly issues with the interpretation of the results, and challenges associated with trying to translate national polls into state results for an Electoral College projection.
There are numerous implications of this interpretation failure for commercial research and analytics on the things that are important to the Market Research industry: trust in research (especially surveys!), new tools and techniques, predicting & modeling behavior or trends, implicit vs. explicit data sources, the application of cognitive & behavioral psychology, and more.
Arguably, approaches using experimental polling methods, social media analytics, behavioral economics-based analysis, “big data”, meta analysis and data synthesis, and text analytics were more predictive of the results than traditional polling – and the implications of that for other forms of research should not be ignored.
However, there is another set of tools that also can help address some of the issues with polling: qualitative research. Sure, campaigns use focus groups all the time, but in the era of communities, mobile ethnography, social media listening, iterative and agile rapid qualitative feedback and many other approaches, the available tool kit has expanded far outside of dial testing in a focus facility.
Dianne Hessan, Founder and former CEO of community pioneer C-Space worked with the Clinton campaign to use more qualitative approaches to provide ongoing insights to the team. After the election she wrote in the Boston Globe a bit about how she used qualitative tools to try to fine tune the messaging:
“Over the summer, I found and interviewed over 300 undecided voters, and 250 of them agreed to stay in touch, to send me weekly diary entries about their emotions, what they were thinking about both Clinton and Trump, and how they were leaning when it came to their vote. I had no responsibility to change their views; instead, I synthesized the data that I was collecting, and reported in to the campaign. I also added the insights that I had and made regular suggestions about how the campaign might better articulate its positions and modify its strategies.”
She goes on to write about how she knew there was a problem: the infamous “basket of deplorables” comment made by Secretary Clinton at a fundraiser. The backlash among the undecided voters she was engaged with was instant and forceful. The damage was done, but using qualitative research she was able to apply the appropriate context and insight to the polling data to help the campaign understand why they were in trouble.
That is the real secret here: quantitative research is a highly scalable and cost effective way to get to who, what, when, where, and how – but not why. To achieve real predictive accuracy, we need a contextual framework that gets us closer to consumers, that allows us to get past the limits of check boxes and grids and instead helps us understand the underlying drivers of behavior. There may come a time when different tools can fill this gap, but today only qualitative approaches can help achieve that.
The issue around missing the human story in the math of quant data isn’t limited just to research though; it impacts basically all of the predictive sciences. Wolfgang Münchau writes in the Financial Times:
“High quality global journalism requires investment. The curse of our time is fake maths. Think of it as fake news for numerically literate intellectuals: it is the abuse of statistics and economic models to peddle one’s own political prejudice… Economic models have their uses, as do opinion polls. They provide information to policymakers and markets. But nobody can see through the fog of the future.”
Qualitative techniques may not entirely clear that fog, but they are lights that make it easier to navigate by illuminating the way into the hearts and minds of humans. Pollsters, pundits and political scientists would do well by looking past the math of polls and surveys and starting to use a variety of qualitative tools to really understand consumers.