Webinar Recording | 6 Steps to Perfect Qual- Step 1: Qualitative Research Design

6 Steps to Perfect Qual – Free L&E Webinar Series

On January 17, 2018, L&E hosted our first webinar in our “Six Steps to Perfect Qual” webinar series for 2018 with Tia Maurer, David F. Harris and Jim White as our panelists. The first step in our series focused on qualitative research design and its importance to great qualitative research.

Miss out on this webinar? Not to fear, you can view the webinar recording here.

To continue reading, download our latest white paper by clicking here (no form to fill out).

L&E Research Announces New Technology Partnerships

Raleigh, NC – August 31, 2017 – L&E Research, the preferred choice for recruiting and facilities in the qualitative market research industry, is pleased to announce its new partnerships with leading technology companies Aha!, IDG (Informed Decisions Group), Isobar and LivingLens. 

“The latest Corporate Researcher Report from Quirk’s suggests there is a considerable gap between the qualitative methodologies clients are curious about and what they’re actually using.  Newer methodologies like virtual reality, biometrics (including neuromarketing), and online qualitative software still face questions from corporate researchers. That gap is continuing to close and companies are looking to find new ways to get answers to their questions better, faster and whenever possible, at a lower cost,” said Brett Watkins, President of L&E Research. “We are excited about these new offerings we can provide to our clients in the methodologies of virtual reality, video analytics, online emotional measurement tools, in-the-moment online tools and L&E’s own 360° streaming.”

“Aha!, IDG, Isobar and LivingLens are the most forward-thinking leaders in qualitative technology, and we’re proud to partner with them to offer our clients a wide array of choices to solve any qualitative research challenge. L&E’s unique team of Research Design Engineers (RDEs) who are trained in the latest qualitative methodologies and the latest technologies used to execute them, help guide clients toward finding the right answers to solve their business problems. With L&E’s combination of talent and technology, we’re a one-stop shop for clients and their qualitative research needs.”

About L&E Research

L&E Research specializes in qualitative research recruitment and provides additional qualitative market research solutions including focus group facilities, online qualitative technologies and more. L&E is headquartered in Raleigh, NC and has been named to the Inc. 5000 list of fastest growing private companies in the U.S. multiple times. In addition to a virtual presence nationwide, L&E has physical office locations in 7 markets, including Cincinnati, OH; Columbus, OH; Charlotte, NC; Minneapolis, MN; Raleigh, NC; St. Louis, MO and Tampa, FL.

Contact:

Michelle Landmesser
Vice President of Sales
L&E Research
919-256-9610
www.leresearch.com

###

AI and Automation

AI and Automation – More Time for the Good Stuff.

AI and Automation.  These are terms that were first associated with big data, then with structured data, and now with small unstructured data (read “qualitative”).  The capabilities of AI and automation are filtering into most aspects of the qualitative process.  And the result is qualitative research that is often cheaper, faster, and – recognizing that this word is subjective – better.  But more importantly, it opens the analyst’s time for the good stuff – solving the client problem.

Here are a few ways our qualitative life is being changed by automation and AI.

  • Sampling – sample providers are using API’s to integrate with qualitative platforms to enable fast, efficient, and cheap access to consumers for a variety of qualitative approaches: communities, IDI’s, online groups and emerging hybrid approaches.
  • Data collection – Online platforms for qualitative and hybrid approaches  allow for quick turn qualitative research unimaginable a few years ago. Need to conduct 12 groups globally in a day? Done.  Need to test concepts with 1000 people and get qualitative feedback in 24 hours? Done.  Need to conduct “micro-communities” for a week on new positioning? Done.
  • Moderation – the emergence of talent marketplaces now gives buyers the ability to become research DJ’s; to mix the right individuals with the right skillsets globally with the right technology to meet the business need in real time and all online. Qualitative research is entering the “Democratization Age”, where technology drives the human elements as much as it does the data collection process.
  • Analysis – advances in AI, text analytics and data visualization tools now allow for the fast (within seconds, literally) categorization, clustering, thematic discovery, emotional analysis and even voice or facial analysis of responses as part of the tool kit. This makes the analysis of transcription, video, image and voice data as easy as running basic descriptive statistics in quantitative data.

It’s important to note that the effectiveness of all these tools are dependent on the right people, talking about the right things, in the right way.  Recruiting, study structure, and moderation are still critical elements.  AI and automation are still only as good as the thoughtful humans directing it. The advancement in these tools are drastically reducing the human time and effort it takes to gather and analyze qualitative information while improving the types of information and volume of information that can be analyzed.

This leaves the analyst with more information to work with and more time to focus on the implications of the insights to address the business issue.  Isn’t that what we really need – more time for the good stuff?

Virtual Reality in Marketing Research

Few developments in technology are as compelling as Virtual Reality (VR). This technology represents a completely unique way to experience life. My first experience with VR was about 4 years ago at a conference. When I put the headset on, I experienced my own personal case study for System 1 and System 2. One part of my brain knew exactly where I was (standing on a floor in a boring conference room in just another conference hotel), the other part knew exactly where I was (60 feet in the air, standing on a small pedestal with no handrails) – and it was two separate places – with two VASTLY different emotions.

It’s probably useful to have a quick definition.

Virtual Reality is a computer-generated simulation of a three-dimensional environment that can be interacted with in a seemingly real or physical way. This is done with a headset and/or gloves with sensors in them. For now, VR is primarily about touch and sight.  Sometimes sound is incorporated but rarely smell or taste.

This technology has become easily accessible in the past year or so as headsets have dropped from $40,000 in 2014 to just a few hundred for a nice headset today. Programming the environment is easier and cheaper (but still not cheap). We are quickly moving from prohibitive to accessible.

The most beneficial application of VR is the ability to interact with something that does not exist or is rare in the real world. Even more practically, the benefit is more focused on things that would have either of two characteristics; a place/thing that is expensive to create or an experience that is rare or nuanced.

A person can walk around the new store layout, without the store. Check the usability of a new hotel room without a hotel room. Sit in the brand new Corvette, without a car. All these are great use cases for VR, in that the cost to develop a virtual store, hotel, or car is way less than a real one.

The next blockbuster use case for VR is ethnography for rare and nuanced experiences. First aid products are one of the more important products that we have in our home, but (thankfully) rarely used. VR can be used to create moments when that product is necessary without anyone actually being hurt. We can then see the reactions and the opportunities to improve products without causing injury.

Another similar use case is identifying triggers for behavior change. As we all know, this is difficult at best and small variations can be the tipping point that creates change. VR can be used to test effectiveness of these programs. In VR, behaviors can be seen to their logical outcomes without the downside of experiencing those outcomes. For example, you can get a “virtual disease” and see the effectiveness of various compliance programs.

There are several other benefits for VR. A quick few are that the experience created can be in the same environment for everyone, and therefore less unexplained variance. Difficult to reach audiences, based on security or health, can be brought into the research process with VR. And other tools, such as facial coding and eye tracking, can be integrated for deeper understanding of the consumer experience.

There are a few things to consider when looking at VR projects.

  • The technology is getting a lot cheaper, but the programming can be quite expensive.
  • The broader the virtual world, the more expensive the programming.
  • The cost/benefit does need to be weighed. Not a lot of people have access to the technology, so the researcher will have to provide access – either through a central location or providing to each person in their location.
  • Be aware of potential health effects. If the environment has situation that could be dangerous in a virtual world, the reactions of the body can be similar to those in the real world.

VR is here to stay and will continue to make inroads into the research process and business issues. It is estimated that the VR market will be $150 Billion within the next 3 years. And with the new programming tools and improved hardware, researchers will be able to do more with faster and greater reach into the consumer brain. Therefore, brands will be able to do more with their product development and marketing dollars, particularly when the target audience is hard to get into a single location.

Webinar Recording | More Insights, Tools and Techniques for Writing Better Screeners

Writing a high-quality screener is critical to getting the right respondents for your qualitative research project. Since screeners are actually short questionnaires, why not apply the same skills and techniques to developing your screeners? On May 12, 2017 L&E hosted a webinar with David F. Harris, author of, The Complete Guide to Writing Questionnaires: How to Get Better Information for Better DecisionsDavid provided detailed examples of how to improve your screeners to get better respondents for your qualitative research.

Webinar Presentation

To download a PDF copy of David’s presentation from the webinar, click here.

Webinar Recording

About David F. Harris

David F. Harris conducts qualitative and quantitative research for companies in a variety of industries. As founder of Insight & Measurement, he also conducts training and consulting on questionnaire design and screener development. He is author of, The Complete Guide to Writing Questionnaires: How to Get Better Information for Better Decisions. He received his B.A. from Reed College, in Portland, Oregon, and his M.A. in Quantitative Psychology from the L. L. Thurstone Psychometric Laboratory at the University of North Carolina at Chapel Hill.

Emotional Measurement in Qualitative Research: Key Considerations

Prefrontal Cortex, Limbic System, System 1, Behavioral Economics, Emotional Quotient – all phrases that didn’t matter in the market research industry until recently (the definition of recently being up for debate). The point is, market researchers have learned that emotions play a critical role in consumer behavior and satisfaction.

Knowing that emotions play a role and knowing which emotions play what role are two very different things. While no one has all the answers to this problem (and all the answers may not even exist), here are some issues to consider when you take on the task of understanding emotions in a business context.

  • Segmentation: It’s not news that people are different from one another. But particularly with emotional measurement, the emotions that people feel about a category, product, or brand are often related to their degree of engagement. For example, heavy users of Apple have a different and deeper emotional connection to the products and brand than moderate users of Apple. In addition to engagement, the emotional nature of the individual person can play a role in the connection to the brand. There are several ‘personality traits’ type models that can help structure the understanding that comes through qualitative research.
  • Context: When delving into the emotions elicited by a category, product, or brand – context can be incredibly important depending on the nature of the product and the situations in which the product is used. Some products are personal by nature in that there is not much social interaction related to the product. Cleaning supplies are a reasonable example in that they are generally used in a private setting (Saturday morning around the house) and not part of a large social effort. In contrast, birthday cakes are almost always used in a social setting – be it large or small. The emotions elicited by either can be generally viewed as the emotions in any situation where these products are used. But let’s take the example of coffee. Coffee is sometimes used in personal settings and sometimes in social settings. The emotions elicited in each can be very different from each other with the exact same product.

  • Conscious: Many of the System 1 approaches would have you believe that all emotions are nonconscious or the decisions made from these emotions are all nonconscious. While this may be true for some people, most can articulate real emotions with a modest degree of depth and accuracy. Happy vs. Sad, Scared vs. Comforted, etc. In situations where nuance is not warranted, in depth interviews (and a good tool kit from a moderator) may be sufficient to uncover the emotions and emotional drivers for a project.
  • Nonconscious – Implicit vs. Biometric Measures: In those cases where nuance is required or the respondent may be either unwilling or unable to understand or articulate, implicit association and biometric feedback are the two categories of tools that offer the most insight. A distinction is made here because definitions of these two words are wavering – Implicit is any tool that delves into nonconscious emotions but does not measure some component of the body in doing so. For (at least our definition of) implicit, the tool kit is generally an implicit association test or metaphor elicitation. There are several good tools and techniques that fall in or near these definitions. Biometric feedback has made important advance in quality and cost over the past few years and is therefore being used by more and more clients. The most common of these tools are eye-tracking and facial coding. Others that are reasonably available for qualitative research include Galvanic Skin Response, EEG, fMRI, and heart rate monitoring. While these are still early in their evolution and there is much to learn, these tools can help researchers understand emotions and emotional triggers.

Emotional measurement is difficult. However, with guidance and tools, it is easier than it ever has been, and these tools make a useful addition to the qualitative researcher’s toolkit to help deliver deeper insights that deliver greater business value.

Qual at Quant Scale

Quantitative and Qualitative – Not Much Longer.

Most that have been in the research industry for more than a couple of years have experienced the following situation as we’ve sat behind the glass with our client:

  • Marketing: Did you hear what that person just said. That’s brilliant. That’s our solution.
  • Researcher: That was good, but it’s just one voice. We are exploring right now. We’ll validate this idea in the next phase.
  • Marketing: I think we have our answer.

This is an overt oversimplification for illustration purposes, but the reality is often not far off point. The problem really comes from the alignment of an idea that has been uncovered in qualitative research and the marketing manager’s predisposition to a particular solution. Add several weeks in order for the manager to advocate before all the results are in – and researchers frequently find that the “answer” has been acted on before the research is completed.

Reducing or eliminating the time from exploration to validation has been a goal of research for many years. We’ve been through quant/qual to explore the results of quantitative research through qualitative exploration with a subset of respondents (or vice versa). Communities offer some of this capability and there have been others as well, but they have all left some gaps.

Several newer types of tools have improved our capabilities to address traditionally qualitative topics at the scale of quantitative research. These tools can be roughly broken into three categories; methodologies, crowd-sourced approaches, or approaches built on artificial intelligence. Several approaches use multiple categories, but are usually dominated by one category.

  • In terms of methodologies, arguably the most important advancements have been made in emotional measurement, particularly implicit. All aspects of a brand should be aligned to a promise, and the execution of the brand experience should be aligned to make good on that promise. Emotional measurement allows us to understand how the brand is aligning itself to the heart of the consumer. Until recently, exploring and validating emotions was not only difficult, but was disconnected from the time it took to do each. New methodologies can do both, at scale, and cost-efficiently. The methodologies are drawn from traditional tools in psychology and marketing research, but adapted for speed and scale by technology.
  • Crowd-sourcing is a tool that has been around for hundreds of years – but has come into its own the past few years. The premise that the crowd has capabilities that a more limited pool does not have is applicable to a number of problems qualitative research has typically addressed. Ideation, concept evaluation, marketing communications, and even competitive assessment can be addressed both qualitatively and quantitatively at the same time through tools like prediction markets, mobile ethnography, and some communities.
  • Artificial intelligence is the newest tool and the one with perhaps the most potential to disrupt the line between exploration and validation. These tools can analyze large amounts of data coming in various forms (images, videos, and text analysis – voice is still a little far from practical as of this posting). The analysis can be near real time – allowing for the automated development of a follow up hypothesis, data gathering and analysis of that hypothesis. And so on. Sound familiar? All of this can be done with hundreds of respondents.

Many of these approaches will change over the next three to five years as technology allows continued improvements in access to people and speed of analysis. Artificial intelligence will have similar changes, primarily in the quality and nuance of the analysis. These improvements, and others happening in the industry, will make the difference between qualitative and quantitative research a false distinction.

The Intersection of Scalable Qual, Mobile and Video

Scaling Qual

One of the most arresting stories of the 21st century is how quickly and profoundly technology has changed marketing. It began innocently enough with banner ads on websites. Then came search and communities. Quickly after, the complex ecosystem of advertising technology (Adtech) – a multi-billion dollar vertical with a cornucopia of offerings stretching across display, search, video, mobile, social, content marketing, native advertising, gaming and commerce – emerged.

The next generation of technology emerging on top of Adtech is marketing technology (Martech). It aims to make marketing “programmatic” – operating automatically and autonomously, similar to programmatic advertising. Martech entered the vernacular relatively recently but has grown rapidly, with companies that deal in it growing from about 100 in 2011 to more than 1,800 by 2015. Martech today is spread across consumer and B2B marketing domains while it increasingly absorbs the elements of Adtech.

What connects and underpins these technologies is data. Behavioral data is generated frequently as the “exhaust” of all digitalized processes. The emergence of this data creates and fuels the disintermediation of (quantitative) research through the rise of business intelligence (BI). BI is a collection of technologies originally developed to look at areas like supply chain or finance; data feeds not relevant to the insights category. But today we are seeing the absorption of all things market research into BI, driven by expansion of data collection processes, visualization platforms or analytical systems that incorporate many of the quantitative tools.

However, the impact on qualitative research tools and platform is different, if not opposite. These technologies make current qual tools faster, better and more efficient, and also help scale them in ways that used to be cost prohibitive, if not entirely impossible, in the offline world.

Some of the new approaches are new interpretations of existing tools, like participant observations, diaries, IDIs, or focus groups. Here technology delivers a significant change but may not be a game changer. Other approaches result in completely new tool – in fact, sometimes it is debatable if these are qual or quant; just think about sentiment analysis in social media.

Once upon a time, the demarcation line between qualitative and quantitative research was clear and impossible to cross. This is no longer the case. Today’s “new qual” maintains the depth and richness of insights from the “old qual” era, but it can also scale in a manner that is more reminiscent of quant.

The Mobile Imperative

While big data and the technologies that create and live off of it are frequently discussed in research, they are many degrees removed from consumers and respondents. The technology that impacted the consumer most in this decade is undoubtedly mobile.

The smartphone has seen the fastest ever consumer adoption rate and half of the world’s population now has a mobile subscription – up from just 1 in 5 ten years ago. It has become our primary connector to the world, offering a solution in just about every situation (to paraphrase the popular ad: “there’s an app for everything”).

This is also true for the insights industry because research and mobile get along well. With all the capabilities mobile offers and the need for the research to be faster and better in data gathering, this dynamic duo presents great opportunities.

Qualitative research particularly benefits from mobile. With enhanced techniques such as ethnography and diaries, some argue that mobile has also created entirely new research methodologies.

Mobile offers a solution for several research challenges, including:

  • Mobile Boards: These apps let respondents to post messages, pictures, and videos to a board and conduct discussions while they are on-the-go and away from their computers. By going mobile, respondents can share their thoughts and experiences wherever life takes them. Many also let participants interact with one another and extend their research community (MROC) experience.
  • Ethnographic Tools: These tools allow researchers to be with people in the moment, so they can understand what they think and feel, as well as see them in their most natural context. With the help of these apps, researchers can follow people’s’ journeys and brand interactions as they happen; be with them in their homes; on their way to work; or at a live event. Researchers can also use the app themselves, during shop-alongs, consumer connects, or for mystery shopping projects.
  • Research Platforms: These are dynamic scheduling platforms that allow tasks to be set up to appear at certain times, or based on certain events, including prior respondent uploads. This means respondents get to unlock their tasks and activities which results in deep engagement with the project, making it fun, game-like and immersive. On the back end, researchers are connected to a platform to capture thoughts, reactions and behaviors as they happen. It lets respondents conveniently record their messages, videos, and pictures, with just one simple click. Many offer maximum data collection flexibility and allow researchers to easily review all information. The benefit is to see how people experience products, services and everyday life in the same moment they do.
  • Mobile Surveys on the Fly: Other apps let researchers create forms and surveys in seconds on a web dashboard then publish them to a native mobile application for mobile respondents to reply instantly and remotely. Frequently “software as service” features include the ability to reply to forms offline, collect signatures and pictures, together with their GPS location, time and date of collection. Most of these features can be custom-integrated to existing information system and other mobile apps.
  • Passive Data Collectors: This class of apps allows researchers to mine the data mobiles collect passively either directly or licensing it from data aggregators. With the help of these tools, you can easily measure content consumption on the device, or string together various behavioral and attitudinal data streams, e.g.: linking a survey response to social media activity.

The Rise of Video

Video took the consumers’ world by storm and now makes it into almost every interaction they have online. Daily time spent watching online video in 2015 grew by 23.3 %  across 40 key markets. And, thanks to the continued popularity of YouTube, the growth of video on Facebook/Instagram, and the accessibility of high definition video on almost any device, video growth will expand by another 19.8% in 2016.

Video increases engagement and consumer involvement in every situation. For example, landing pages with video can lead to 80% more conversions. In fact, 88% of visitors stay longer on a site with prominent video displayed. Those that stay longer spend an average of 120 seconds more on a retail site and are 64% more likely to purchase after viewing a single product video.

With sophisticated mobile devices, network improvements and consumers’ increasing ease at sharing their life via mobile, it’s no wonder that half of all video traffic in the world is now routed to a mobile phone or tablet.

While it began with professionally produced content being scaled up, the proliferation of mobile devices shifted this balance towards user-generated content. This helped create new companies like GoPro, communication services like Skype or live video feeds for social media networks like Meerkat.

This has meant that video analytics have also grown at a comparable rate. The need for enhanced insights from customer behavior gathered by the existing video platforms is driving improvements in these analytical solutions.

The video analytics market broadly falls into categories including security management, crowd detection, pattern recognition, and other applications. Pattern recognition includes face recognition, Optical Character Recognition (OCR), and object detection. This category, partly driven by the quest for insights, is experiencing the greatest increase in demand enhancing the economical scalability of video based insights generation solutions.

When Mobile, Video and Scalable Qual Meet

Scalable qual, mobile and video are powerful forces on their own, but when you combine them together a truly formidable research proposition emerges.

  • Richness & Depth of Insight: The relationship consumers have with their mobile phones is one of the most intimate. These devices are the connectors and conduits between the outside world and us. We share every moment on our mobile phones, sometime perhaps even too much. But for researchers, mobile phones are a treasure trove of rich, deep, descriptive, explanatory, evocative and inspiring insights. The combination of capturing behavioral and attitudinal data along with the context at the same time is truly unique.
  • Agility & Speed: Today these changes have moved from “how quickly can you do it” to “can the speed of your decision-making match the speed of my insight delivery?” The in-the-moment nature of mobile platforms not only enables speed but practically dictates it. Equally importantly is now there is no need for “campaign thinking.” Consumers’ expectations and communication protocols are driven by the “chatty nature” of mobile. The primary case and fastest growing mobile applications are the chat platforms. These are setting the tone by establishing precedence and driving preference. The asynchronous nature of these platforms allows researchers to pick up conversations where they were left off, matching consumers’ natural behaviors on the platform.
  • Scalability/Sample Sizes: As discussed earlier, new emerging qual platforms approach ‘size’ through scalability. In other words, gone are times when one had to lock in on approaches driven by sample sizes. Mobile video delivers rich and deep insight with sample sizes ranging from “qual” to “quant” coupled with the right technology. A platform that can effortlessly scale across research disciplines makes a viable tool for “small data” research projects or larger scale analytics.
  • Socializing the Result: A research project is only as good as how effectively it can inform and persuade business decision makers. Video’s magic also works here. More than 80% of senior executives watch more video than they did a year ago, and 75% of executives are watching work-related videos every week. Given the choice, 59% of executives would rather watch a video than read an article, making video an effective and efficient presentation tool.

In summary, mobile video is an important addition to the researcher’s toolbox, upending the traditional qual-quant equation by delivering deep, rich, insightful information at unprecedented scale and speed. As techniques and methods further mature there is less and less excuse for not using it in your next research project.

L&E Webinar Recording | Automation: Assessing the Impact on Qualitative Research

Automation is set to be a game changer for market research in 2017, a view supported by the recent GRIT report which listed it as one of the top factors likely to disrupt ‘business as usual’ in 2017. Most of the discussion about Automation has focused, so far, on its impact in quantitative areas, such as survey design, analysis, and big data. But Automation is also threatening to disrupt, challenge, and change the way many aspects of qual are conducted.

On February 17, 2017, L&E hosted a webinar, “Automation: Assessing the Impact on Qualitative Research”, with guest speaker Ray Poynter, Founder of NewMR. In this webinar, Ray highlighted the key ways that automation will impact qualitative research and provided a list of tips and action points that will help you stay ahead of the game in this year of change.

Webinar Video

Webinar Presentation

To download a PDF of Ray’s presentation from the webinar, click here.

About Ray Poynter

Ray has spent the last thirty-five years at the intersection of innovation, technology, and market research. Ray is the Managing Director of The Future Place and is the author of The Handbook of Mobile Market ResearchThe Handbook of Online and Social Media Research, and editor of ESOMAR’s book Answers to Contemporary Market Research Questions. Ray is in popular demand as a conference speaker, workshop leader, writer and consultant, appearing regularly in Europe, North America, and Asia Pacific.

Using Qualitative Research to Prevent Another Election Debacle

Much has been written about, discussed, and broadcast about the apparent failure of public opinion and political polling in 2017.

To be clear, there was not an issue with the data in the U.S. election; almost all reputable polls were within the margin of error nationally. There were certainly issues with the interpretation of the results, and challenges associated with trying to translate national polls into state results for an Electoral College projection.

There are numerous implications of this interpretation failure for commercial research and analytics on the things that are important to the Market Research industry: trust in research (especially surveys!), new tools and techniques, predicting & modeling behavior or trends, implicit vs. explicit data sources, the application of cognitive & behavioral psychology, and more.

Arguably, approaches using experimental polling methodssocial media analyticsbehavioral economics-based analysis“big data”meta analysis and data synthesis, and text analytics were more predictive of the results than traditional polling – and the implications of that for other forms of research should not be ignored.

However, there is another set of tools that also can help address some of the issues with polling: qualitative research. Sure, campaigns use focus groups all the time, but in the era of communities, mobile ethnography, social media listening, iterative and agile rapid qualitative feedback and many other approaches, the available tool kit has expanded far outside of dial testing in a focus facility.

Dianne Hessan, Founder and former CEO of community pioneer C-Space worked with the Clinton campaign to use more qualitative approaches to provide ongoing insights to the team. After the election she wrote in the Boston Globe a bit about how she used qualitative tools to try to fine tune the messaging:

“Over the summer, I found and interviewed over 300 undecided voters, and 250 of them agreed to stay in touch, to send me weekly diary entries about their emotions, what they were thinking about both Clinton and Trump, and how they were leaning when it came to their vote. I had no responsibility to change their views; instead, I synthesized the data that I was collecting, and reported in to the campaign. I also added the insights that I had and made regular suggestions about how the campaign might better articulate its positions and modify its strategies.”

She goes on to write about how she knew there was a problem: the infamous “basket of deplorables” comment made by Secretary Clinton at a fundraiser. The backlash among the undecided voters she was engaged with was instant and forceful. The damage was done, but using qualitative research she was able to apply the appropriate context and insight to the polling data to help the campaign understand why they were in trouble.

That is the real secret here: quantitative research is a highly scalable and cost effective way to get to who, what, when, where, and how – but not why. To achieve real predictive accuracy, we need a contextual framework that gets us closer to consumers, that allows us to get past the limits of check boxes and grids and instead helps us understand the underlying drivers of behavior. There may come a time when different tools can fill this gap, but today only qualitative approaches can help achieve that.

The issue around missing the human story in the math of quant data isn’t limited just to research though; it impacts basically all of the predictive sciences. Wolfgang Münchau writes in the Financial Times:

“High quality global journalism requires investment. The curse of our time is fake maths. Think of it as fake news for numerically literate intellectuals: it is the abuse of statistics and economic models to peddle one’s own political prejudice… Economic models have their uses, as do opinion polls. They provide information to policymakers and markets. But nobody can see through the fog of the future.”

Qualitative techniques may not entirely clear that fog, but they are lights that make it easier to navigate by illuminating the way into the hearts and minds of humans. Pollsters, pundits and political scientists would do well by looking past the math of polls and surveys and starting to use a variety of qualitative tools to really understand consumers.