Technology Archives - Page 2 of 5

Recipe for the perfect qual panel: Step Four

DWG Admin on September 22, 2023

Recipe for the perfect qual panel

Step Four: Recruitment at Its Best

So, you’ve assembled your perfect panel, what now? Now, it’s time to ensure that your panel not only reflects the diversity of the groups you’re studying but also guarantees the accuracy and clarity of the information they provide.

In this crucial step, we look at how to master accuracy in programming, employ dual-verification and authentication processes, track fraudulent panelists, and closely monitor participation to maintain the highest standards of research quality.

Mastering accuracy in programming

When recruiting your dream team of potential research participants, a well-crafted screener is crucial; it can make or break your research project, and the process of creating one that satisfies all parties involved can be a complex and time-consuming endeavor. Gone are the days when screeners were administered via paper and pencil; technological advancements have taken their place. We now harness the power of survey software for screening, which brings efficiency and precision to the forefront.

However, the path to a flawless screener is not without its challenges. Without multiple layers of quality and accuracy checks during the programming process, there’s a lurking risk of errors that could lead to disastrous misrecruits.

We have implemented rigorous programming metrics to ensure your carefully crafted screener is executed flawlessly. Our process has an initial goal of achieving 92%+ accuracy; to which follows a quality check to achieve 98% + accuracy. All screeners are programmed within 24 hours, with 95% or more meeting this deadline.

Ensuring Rock-Solid Consistency in Responses

This is another cornerstone of a recruitment process. Screening recruits both digitally and verbally is important for maximum accuracy. We begin by filtering candidates who meet essential criteria such as gender, age, education, income levels, vehicle or homeownership status. Our proprietary panel software tracks over a hundred data points so that we can precisely target the audience, therefore reducing panel fatigue.

Once we have a broad list of potential candidates, we email them a pre-screener with core qualifying questions. This helps us narrow down the initial list to a more manageable size. Our recruiting team then contacts those who meet the pre-screening criteria by phone. They ask the same core questions a second time to confirm consistency in responses.

Tracking Fraudulent Panelists

The success of a research project depends on many variables, but respondent quality is the most important one. Recruit validation isn’t simply about verifying screening criteria and quotas; recruiters must also confirm the recruit is providing honest responses.

It’s becoming increasingly common for fraudulent panelists to deceive researchers by submitting inaccurate information. It’s important to fight back by enabling safeguards to flag and ban dishonest panelists from future research. Including “red herring” questions in screeners to conceal correct responses. Prior to initiating the screening process, we collaborate closely with our clients to identify and address any potentially biased or leading screening questions. This collaborative effort is key in mitigating the risk of fraudulent panelists. Our recruiters and project managers carefully review screening responses to identify any potential warning signs, such as panelists who frequently select all options at the highest frequency. For instance, it’s highly unlikely that someone ordered 97 sandwiches from a restaurant last week. (To give you an idea, this is a real example of an absurd response that we came across a few years ago!)

Dealing with professional respondents

In addition to excluding fraudulent panelists, there’s a danger you’ll run into a “professional respondent”, those who provide honest responses but have participated in research too many times. Ideally, consumer panelists should participate in 1-2 research studies in a calendar year and, at most, 10-12 research studies ever. If panelists are recycled too frequently, their responses may lose the fresh perspective needed; in other words, they may subconsciously or inadvertently become familiar with the process of a research interview and may begin to answer questions differently based on their research experience. Participation is limited to conform to industry-standard parameters, and previous research is monitored. Additionally, we have implemented a recruiting metric that aims to ensure at least 30% of first-time respondents.

Crafting effective screeners, ensuring response consistency, and safeguarding against fraudulent panelists are key steps to build a high-quality research panel. In your research journey, remember that quality remains the bedrock of impactful insights.

6th Annual Future Trends of Market Research and Technology

DWG Admin on August 29, 2023

For the 6th consecutive year, industry experts from Microsoft, Procter & Gamble, Greenbook, and L&E Research gathered to discuss the state of the market research industry and its challenges, particularly in relation to data quality and authenticity.

Generative AI has been the hottest topic of conversation throughout most of 2023, so we jumped right into it, head first. One might suspect that the discussion would be focused on the unethical use of generative AI when research participants are completing surveys and screeners, but Lenny Murphy shared an example that was far more nefarious. When analyzing the latest results from the annual GRIT survey, the Greenbook team identified over 600 responses that had been AI generated, which comprised almost 20% of their overall survey completions. Suppliers should have techniques and processes in place to mitigate respondent dishonesty, but what we didn’t expect to see was fraud committed by organizations within our industry. Panelists who attempt to defraud research do so primarily for one reason: the incentive. Although the GRIT survey is not incentivized, Charlie called attention to the fact that the GRIT results influence corporate budgets and the technology solutions that clients look to engage with. So by training AI to auto-complete the survey, these companies are attempting to skew the results in their favor to increase their business opportunities in an incredibly unethical manner. In fact, this level of deception is more appalling than a consumer participant trying to make a quick $75 bucks. The greatest challenge and concern is that the AI-generated responses were difficult to distinguish due to their contextual accuracy and depth.

Continuing the conversation around data quality, Lenny then called attention to fraudulent or inauthentic participant responses within the industry. He mentioned that around 70% of sample responses might be discarded due to authenticity concerns, and shared that we have an urgent need to address the crisis of data quality and authenticity within the market research industry.

What are some things research buyers and designers can do to improve the quality of research collected? Lenny, Barry, and Charlie all made suggestions that may help to mitigate fraudulent data, such as:

Embedding non-conscious measurement in surveys to ensure responses are genuine
Implementing measures to ensure data quality before surveys are even conducted, using metrics and quality assurance steps to detect potential fraud and improve data integrity
Employ strategies like red herring questions and other techniques to identify and remove fraudulent responses
Shift focus away from the race-to-the-bottom pricing mentality and instead prioritize high-quality sample providers with whom trust has been established

Generative AI is a powerful tool, and will do a lot of good. Better algorithm and hardware development has led to more processing power, and manufacturers of neurotechnology devices are making rapid advancements as it relates to scans and wearable devices. Device manufacturers are looking for ways to implement AI sensors into wearables, such as earbuds, smartwatches, VR headsets, and other consumer based technologies. While this technology is still in its infancy, and currently these devices can only measure and pick up low frequencies, the technology will get better, stronger, faster. This will allow practitioners, specifically in the medical field, to monitor brain functions in ways that we probably have not yet realized. Humans have limitations, but computers can process and execute millions of data points and provide analysis in a fraction of the time.

Device manufacturers have also started exploring ways to use devices to regulate moods. For example, an implant can suppress depressive thoughts, like a pacemaker for the brain, therefore enhancing the quality of life for those who suffer from clinical depression. There may come a day when an implanted brain chip could control devices, which could be life-changing to someone who is paraplegic and unable to do so using their own extremities. Modern AI utilizes neural networks modeled after biological brains, and AI is creating, or rather, generating, more like the human brain would. In our industry, researchers are already using AI tools to analyze, synthesize, summarize, and to some degree, predict human behavior, leading to insights, but will there ever be a time when we can use brain scans to gain those same insights? Will our understanding of the human brain, supported by AI, ever advance to the point where we can simply look at a brain map and extract the “why” behind consumer behavior? At its current stage of development, AI’s potential lies in aiding human researchers, and should be treated as an assistant or a new hire, requiring careful review and validation of its outputs by human researchers.

All this, of course, brings up a very important question: Whose job is it to regulate AI development? According to a survey published earlier this year by Ipsos, survey participants say 53% believe it’s the company’s responsibility, and 44% believe it’s the government’s responsibility. Regardless of who makes decisions regarding AI regulation, we have to make good choices early – we won’t be able to backtrack once the proverbial cat is out of the bag.

AI is going to create new challenges that all panel suppliers will need to recognize and resolve. As generative AI gains popularity with the general public, we are likely going to see an increase in use of AI when answering open-ended question types. Articulation is how we gauge the level of responsiveness and engagement when searching for the best potential participants. If their articulation response is generated by AI, the candidate may look good “on paper”, but it’s likely the articulation is false and may not be truly indicative of a quality participant. Another challenge that we’re likely on the precipice of is participants using AI to complete asynchronous research activities. Safeguards on the technology side need to be implemented, such as fraud detection tools, so that researchers are alerted to suspicious responses that may be AI generated.

This does increase the need for honest and transparent communication. When situations such as these arise, clients should be extra diligent in sharing these details with their project manager. Suppliers don’t want to provide clients with panelists who are only interested in shortcuts to a quick incentive, and clients aren’t interested in these candidates either. This must be a collaborative effort of “tattle-tale” to make sure the bad actors are not used for other research.

The key will be understanding what motivates participants, taking steps to work to better the participant’s experience, and for all parties involved to collaborate and embrace best practices as it relates to participant experience. This needs buy-in from everyone: brands, agencies, researchers, technology providers, and panel providers.

So what are some things the industry can do to increase overall panel ecosystem health?

Write better/concise screeners: A screener should have one objective: to qualify or disqualify a recruit. When scripting recruitment screeners, there’s a simple rule that few follow, but all should. If the question isn’t determining an inclusion criteria or a quota, then it should not be on the screener. This is not an opportunity to do quant work, and the time for gathering information is in the research session, not during the qualification phase. What should a supplier do when extraneous questions are included, but the researcher insists “the client wants to know” in advance? A good supplier will make recommendations that are beneficial to all parties involved. A supplier has a responsibility to both their client and the participants with which they engage, so they must find the right solution to both ensure ecosystem health and fulfill the researcher’s objectives. Recommend collecting “profiling” questions in a re-screener, right before the session begins, or in the session itself. Alternatively, a “Getting to know you” assignment completed asynchronously after the qualification phase is a great way to minimize screening while still collecting data prior to the session.
Target the audience: A strong provider will have a powerful system designed to account for a multitude of participant variables, such as the ratio of screeners completed vs screeners qualified, or when someone last qualified for a study, and of course, the ability to track static data points, such as gender, age, ethnicity, etc. Accounting for these variables, the project manager will always target the ideal candidates first, therefore minimizing the overall outreach volume needed to fulfill recruitment, ergo reducing overall screener fatigue on their panel.
Balance your audience with high incidence targets: Sometimes there’s a disconnect between the audience a brand wants to engage for their research and what is viable for panel suppliers to provide. I’m not implying brands shouldn’t try to engage with their low incidence audiences – these customers may be key to finding the insights necessary. However, projects can be very successful when conducted with a broad and balanced audience including both low and high incidence targets, for example brand users (low) vs category users (high). If the recruitment supplier is unable to secure the expected net for a low incidence audience, and options have become scarce, the researcher should be ready to offer flexibility and adapt their research. While this pivot does require additional steps, for example, updating the discussion guide to target an alternate audience, the researcher may find insights they weren’t expecting by engaging an audience they weren’t considering, and the adaptation could be an incredibly successful step in meeting their research objectives.
Create a custom panel with your supplier: Many clients rely on a panel built for their unique research needs. A few examples…
A researcher would like to conduct a series of interviews with a broad target audience, but each phase of research may be targeted, so a custom panel is built implementing the overarching criteria of that audience. From there, the research team can strategically apply a few screening questions to determine which recruits may be the best fit for a variety of research phases. Breaking the screening process into separate conversations reduces panel fatigue because inclusion in the custom panel means that the candidate will be participating in some variation of this large multi-phase project, and they don’t feel like they’ve wasted their time on a qualification process for which they aren’t eligible. In fact this may make them eligible to participate in a few phases of this large project, making this a much more satisfactory and lucrative process to the research participant.
Another example is utilizing a qualitative panel to conduct quant-qual hybrid research recruitment. This can be a very cost-effective solution for projects that require the highest quality for both phases. Your recruitment provider can supply a targeted sample for your quant survey deployment, and based on the responses received, qualitative recruits can be picked from this larger pool (or vice versa).
Compensate fairly: Treating participants respectfully means fair compensation for their time. While budget is always a variable, trying to skimp on participant incentive isn’t where researchers should trim costs. This is bad for the overall ecosystem, and will reduce response rates for your project. Low incidence paired with a poor incentive may mean the recruits that are willing to participate aren’t of the highest caliber – you may get what you pay for.
If the incentive budget is restrictive, find other ways to compensate participants. Are you testing a product, and if so, would it be an option to allow the tester to keep the product? Allowing them to keep a device with a market value of $200 will be an alluring incentive when combined with a less than alluring cash incentive.
Share “bad actors” and validate panels: If we identify that “Joe Schmo” has been cheating the system, then it’s our job to put a stop to that. We may have recruited Joe for a study, only to see his face again three weeks later when he was recruited by a competitor to participate in a study held at our facility. Joe has clearly falsified his last participation, so removing him is beneficial to us, to our competitor, and most importantly, the brands who trust us to provide honest recruits. It’s also beneficial to the panel ecosystem, as he could have been taking a seat away from someone else who honestly and legitimately qualified. Of course the challenge is maintaining protection of PII – how do we share bad actors while still maintaining industry standard security measures? Who manages the list of bad actors? Further complications arise when bad actors are often deceptive enough to not use their real identity, so our “Joe Schmo” could be our competitor’s “Mo Schmo”.

Our industry organizations cater to research buyers, providing information and helping them to navigate emerging technologies and suppliers that can provide solutions necessary to amplify their research. Industry organizations have started to explore better practices as it relates to data quality, diversity and inclusion, and the panel ecosystem, but more should be done as it relates to engaging audiences to participate in research. In short, we need PR for our industry. How do we increase information that market research is not a scam? This is the natural reaction to project outreach when expanding efforts beyond existing opt-in panels. Furthermore, how do we expand our reach about what market research is, how it is beneficial to individuals and brand development, and how both parties play a role in that process? This responsibility has been managed by panel providers, but as we discuss how the industry should collaborate, perhaps it’s time to raise awareness and explore ways all stakeholders can positively contribute to better recruiting practices and better participant management.

As the industry continues to shine a light on data quality issues, it’s important to remember that every little action counts. We all have a role to play as it relates to bettering the panelist market research experience, creating sustainable solutions, and making constant adaptations to maintain the credibility of research outcomes.

We hope you found this summary to be helpful! If you didn’t register for this webinar you can watch this webinar in its entirety by clicking here.

Be on the lookout for our next webinar, which will be in Winter 2024. If you can’t wait until then, you can always view our on-demand webinars. Don’t forget to join our mailing list so you can keep up with what is happening at L&E!

Recipe for the perfect qual panel: Step Three

DWG Admin on July 28, 2023

Recipe for the perfect qual panel

Step Three: Quality Qualitative panels

This is probably one of the most important ingredients in our recipe: the panel. After exploring the planning phase and looking at the importance of good project management, we’re now at the key stage of finding and engaging with the perfect panel for best outcomes.

Finding the right panelists

Quality recruitment brings quality research, and this is why it’s important to have the support of an agency that knows how to build panels specifically for qualitative research, that understands consumer behavior, and knows how to engage respondents. Recruiting through companies who aren’t specialized might bring participants who are not prepared for the depth of answers, or length of surveys that qualitative studies require.

Panelists need to be willing, and honest, and need to be prepared for the tasks they will need to complete. It is critical to only use data from participants that meet these criteria, and your agency partner should always abide by best practices for removing suspicious responses. Completing qualitative research without these processes can waste time and money and is likely to leave companies without a clear direction about key business decisions.

A big community is also important to find the perfect target audience for your research project. The volume of available panelists provides an array of behaviors and opinions, and the feedback necessary for your research objectives. For example, our nationwide qualitative panel has more than 1.25 million members. Some projects might also require custom panels, especially when seeking a low incidence audience that is challenging to find. For example, in the healthcare space a custom panel can be critical when the product or service caters to a niche target, such as a patient with a rare condition or a highly specialized professional.

How to reach them

In a digital world we can utilize a variety of outreach methods to both grow our nationwide panel and to target specific audiences for current projects. Different channels can have different benefits and bring different results; for example targeting via social media directly is often more efficient in prescreening, saving time and money while searching for qualified participants. Also, some niche groups are very difficult to find using traditional methods. With social media it’s easier to access those hard-to-reach respondents hiding in online support forums and communities featuring specific interests.

Posting ads in newsletters, newspapers, radio, television, etc. can also be an effective outreach strategy for select audiences.

Keep them engaged

Keeping panelists engaged is as important as finding the right people. So, how do you do it? First, understand the research participants’ motivations to maintain engagement; while a monetary incentive is key, there are other motivating factors, such as feeling like a part of the community, seeing new products before anyone else, or simply wanting to be heard and share their opinions. We know that strong communication throughout the life of a project is also critical to set expectations, and it should be combined with a well-planned project that keeps panelists engaged to avoid losing them along the way.

So, you should watch out for the factors that could lead to research and screening disengagement: a poorly designed or overly lengthy survey, challenges with the selected research platform, or external variables like distracting surroundings.

Having a panel readily available that has been built specifically for a long-term engagement can lead to quick insights. This requires a meticulously well-planned project from the onset, with parameters for activities defined so that expectations are accurately communicated over the lifecycle of the project. A successful project means stronger trust, which leads to a higher retention rate and deeper level of engagement.

Completing this stage of finding the right panel and keeping them engaged is extremely important for the success of the project. That’s why it is key to find the right partners with the right experience, and a big enough community to be able to provide you with the right audience and quality outcomes.

Recipe for the perfect qual panel: Step 2

DWG Admin on June 30, 2023

Recipe for the perfect qual panel

Step Two: Project Management

Now that you know how to build solid foundations through a successful planning phase, the next step in recruiting the perfect qual panel is project management. The main ingredient at this stage is a good Project Manager! They are trained professionals who can directly manage participants and their engagement, ensuring the success of a project even when participants are falling behind.

So, how can you ensure your project is managed successfully? Here are some tips:

The Kickoff Meeting

Start by scheduling a kickoff meeting. In this meeting, you should cover all the project details: an in-depth screener review including recruitment quotas, recruiting timelines, homework activities, technologies used, and many other variables that may apply to the project. This is a key step to ensure that everyone is on the same page and that the project is set up for success.

Your Team

On top of the Project Manager, as your single point of contact, each project should always have a team of dedicated recruiters that will work on your project until recruitment has been completed. The benefit of this is consistency – if the same team is involved from the beginning, they will understand the nuances of the project and how to best recruit for it. As the relationship develops, your assigned team will be able to anticipate project needs and streamline processes based on their experience with past research, especially when target audiences or research methodologies across studies are similar.

Quality Check

In a well-managed project, each day, all newly secured recruits are scrutinized by the project manager to ensure that they not only meet screening criteria, but that they also meet the quota distribution requested. During their daily review, the Project Manager evaluates other project variables, such as the remaining volume of pre-screened sample, marketing budget amounts, and recruiting hours allocated to verify that the project is still on track.

Be In The Loop

A key role for the project manager is to keep the client in the loop with regular updates on the progress, timelines, and addressing any potential challenges that may arise. At L&E we also offer access to a client portal where you can monitor, track, and share the status of recruitment at any time. New recruits are updated in real time, so you can have the most up to date information.

Having an expert Project Manager and a good recruitment team in place can make a real difference on the outcome of your work. Make sure that you find a trusted partner who’s an expert, organized, spends time briefing the team on what they need to know, and keeps an open line of communication to have a good understanding of progress, timings, and potential challenges.

Unleashing the Power of MROCs to Achieve Better Business Insights

DWG Admin on June 6, 2023

Unleashing the Power of MROCs to Achieve Better Business Insights

The concept of research has evolved significantly, moving towards a hybrid approach that combines qualitative and quantitative methods. This shift has been influenced by various factors, with COVID-19 being a significant game-changer. The pandemic forced in-person research to come to a halt, leading to the rise of online and digital methodologies. The integration of platforms like Zoom allowed for asynchronous studies, where participants engage in activities over a period of time and then follow up with in-depth interviews or focus groups. This hybrid approach has become the norm, with simultaneous or sequential qualitative and quantitative studies being conducted.

Best practices in this new landscape involve leveraging the strengths of both asynchronous and live engagement. Asynchronous methods offer a less intrusive and more convenient way for participants to share their experiences, while live interactions build on the insights gained from the initial asynchronous phase. The hybrid approach also allows for larger teams to be involved in research without overwhelming participants, as the digital aspect provides a more private and comfortable environment. The cost benefits of digital research and the ability to immerse larger audiences in the process are additional advantages of this approach. Overall, the hybrid method enhances the research experience, allowing for deeper insights and more focused discussions.

It’s important to recognize the role of technology, particularly in hybrid activity communities. Video-based interactions and projective techniques, such as storytelling and collage tools, are effective methods for obtaining in-depth insights and emotional context from participants.

Artificial intelligence (AI) tools can assist researchers in analyzing and aggregating data, providing summaries, and generating insights. However, there is a consensus that AI cannot replace the human touch and expertise in research. While AI has its benefits in processing large amounts of data, the interpretation and synthesis of information still require the strategic thinking and understanding of a skilled researcher. AI can enhance efficiency in research but cannot replace the critical thinking and storytelling abilities of human researchers. The human element, empathy, and understanding of human behavior remain essential in delivering valuable insights to clients. Overall, there are both benefits and challenges when using AI in market research, but transparency, communication, and a human-centric approach in leveraging these technologies is key.

When using research technology platforms to collect data, it’s crucial to have a strong support system in place. Brand researchers, recruitment providers, and research panelists will all engage with the platform. Humans naturally have a varied level of comfort when engaging with technology, so it’s important to have a strong support team of humans that can guide each audience. While an instructional video would suffice in many cases, if users are challenged by technology, lost insights may be the unnecessary cost.

Having the right partners to support recruitment is equally important. An experienced partner can make methodological recommendations to successfully execute research. They are experts at managing the logistical needs, maintaining transparency and communication, and setting expectations with panelists throughout the process. Good partners will advise researchers on best practices applicable to a variety of approaches, including: the necessity of a kickoff call, minimizing extraneous touchpoints, avoiding changes to research expectations, and prompt communication with recruits. Calibrating incentives to match the tasks involved will promote engagement and discourage participant drop-off. Granting access to project managers in technology platforms is suggested to monitor participation and handle logistical demands, allowing researchers to focus on the research.

The importance of balancing cost, speed, and quality in research projects is critical, yet often difficult to achieve. Quantitative research can be faster and cheaper but qualitative research does not have to be sacrificed. The application of mixed methods, utilizing both synchronous (such as video interviews) and asynchronous (such as activity-based research) approaches, increases the potential of successfully balancing all variables to meet research objectives.

Hybrid research approaches are a great solution when both quantitative and qualitative research are necessary. Hybrid approaches can offer the trifecta of cost, quality, and speed. For example, participants are screened and qualified for a panel with a simple quantitative survey as the first activity. Screeners should always be concise and only include qualifying questions, but using the quant survey to both collect data and expand screening can allow researchers to narrow the participant pool in a very deliberate manner prior to the qualitative phases, selecting only the candidates that are most appropriate based on the activities expected. Costs are low due to a single recruiting fee and minimal recontacting. Quality is high as responses to previous phases can be leveraged to select the strongest candidates for a variety of activity types. Speed is increased since panelists can be quickly selected from an engaged pool, rather than the need to secure new candidates for each phase.

We hope you found this summary to be helpful! If you didn’t register for this webinar you can watch this webinar in its entirety by clicking here.

Be on the lookout for our next webinar, which will be in August 2023. If you can’t wait until then, you can always view our on-demand webinars. Don’t forget to join our mailing list so you can keep up with what is happening at L&E!

A Prescription for Better Research in 2023: How to Make In-Person Healthcare Research Safe and Effective

DWG Admin on February 7, 2023

A Prescription for Better Research in 2023: How to Make In-Person Healthcare Research Safe and Effective

When the pandemic began in March of 2020, our industry had to (quickly!) figure out how to continue conducting research when stay-at-home orders were implemented. Researchers had to adapt their methodologies to be more inclusive of remote options. In our latest webinar, we discussed these necessary modifications to processes and methodologies with researchers in the healthcare space. L&E Research, along with many companies, had the good fortune of being remotely based for years prior to 2020, so existing processes and metrics were already in place, but we promptly realized that the fortunate were in the minority. Many companies operated from an office or a central location, and had to adapt their internal or operational processes for remote work. Not only was there an adaptation to their internal processes, but researchers who conducted in-person research had to additionally adapt the research process. On the other hand, some researchers were already conducting remote research, so their modifications were minimal. Those who were working remotely and conducting research remotely had a much easier transition than those who had to change both internal and research processes. During our webinar we had the opportunity to talk with two researchers who had very different perspectives: one was conducting almost all research remotely already so the changes made were more about strengthening internal processes to increase efficiency, rather than how to continue the work; the other researcher was conducting almost all research in-person and had to make significant leaps to adapt their processes. The pandemic temporarily halted in-person research, but that halt was the catalyst behind a great deal of innovation in qualitative research. New processes were developed based on the research needed.

When travel bans were implemented for many researchers, but in-person research still needed to be conducted, we worked with our clients to develop and perfect a process that we call Remote Moderation. Facility staff are trained to be the client’s “boots on the ground” when the researcher cannot be in the same physical space as the participant. In addition to the usual services you would expect from a facility partner, we ensure paperwork is completed, we reset the room, often including the device being tested, and can instruct or direct the participant when necessary. The moderator conducts the session remotely via a video conferencing platform, without the need for travel.
Home Use Testing (HUT or IHUT) was frequently used to replace in-person usability testing when possible. Traditionally home use testing is conducted by participants coming into a facility to pick up a product, then test it from their own home for a predetermined amount of time. During the pandemic, we shipped test devices when that was an option for our clients, which eliminated the need for face-to-face contact. Of course, some products or devices are ineligible for HUT due to intellectual property, size limitations, or price limitations, and those still need to be conducted in-person.
After the initial stay-at-home orders were lifted in early summer of 2020, in-person research started to slowly return. Many clients opted to conduct one-on-one interviews rather than focus groups to limit the risk of possible viral transmission, which reduced the amount of people in the facility at any given time, making it safer for both clients and facility staff. In addition to reducing the head count in our facility, we implemented multiple improvements to minimize contact, such as electronic sign-in and check-out processes, including incentive distribution via e-gift card, limiting people in shared spaces (i.e. asking them to wait in their car rather than a lobby), mask mandates, touch-free doors, electrostatic sprayers, plexiglass dividers, mandatory surface sanitization between sessions, and more. In retrospect, we may have gone overboard, but the modifications decreased the risk of transmission, and everyone involved felt like their safety was the number one priority.

There were some misconceptions about healthcare provider recruitment during the pandemic. Many clients delayed their research due to an incorrect assumption that physicians would not be available for research. This was not indicative of our recruiting experience. Acute care physicians and practitioners were definitely strained during the pandemic, but non-emergency service providers experienced a slowdown. Because of stay-at-home orders, many elective procedures were canceled or postponed, so some practitioners experienced a decrease in work over the course of the pandemic. We saw a higher response rate within our healthcare panel because practitioners suddenly had a lot of free time on their hands and wanted to participate in market research, especially and specifically, remote-based research. There was an increase of fraudulent behavior, so recruiters had to be very diligent about vetting recruits, especially in the healthcare space, since incentives are higher, therefore more alluring. Healthcare professionals were validated via online resources to confirm that they were in the profession they stated, and patients had to provide proof of prescription, diagnosis, and/or therapies used before being scheduled for sessions. Now that we are on the other side of the pandemic…or at least moving toward it, what do we anticipate for the future? At the core, we’ve learned that we have to be flexible and we have to adapt. Review internal processes to see if there’s room to streamline or reduce costs. Research should be fit-for-purpose, so if it can be effectively conducted remotely, why would you want to spend more? When it comes to device usability or sensory testing, in-person research isn’t going away. Strive to find a balance and use both remote and in-person methodologies, as there will always be value in utilizing a variety of strategies. We hope you found this summary to be helpful! If you didn’t register for this webinar you can watch this webinar in its entirety by clicking here. Be on the lookout for our next webinar, which will be in Spring of 2023. If you can’t wait until then, you can always view our on-demand webinars. Don’t forget to join our mailing list so you can keep up with what is happening at L&E!

Summary: 5th Annual Future Trends of Market Research and Technology

DWG Admin on November 4, 2022

For the 5th consecutive year, industry experts from Microsoft, Procter & Gamble, Greenbook, and L&E Research gathered to discuss the future trends of market research and technology. The most recent GRIT report shared that the top two unmet needs reported by buyers is technology and sample, so it’s no coincidence that this year the primary topics of conversation were centered around technology and sample.

Technological innovations in qualtech have slowed in the past year as companies seem to be focusing on profitability over growth and innovation. There has been a lot of consolidation within the MRtech ecosystem, combining platforms and solutions to offer more flexibility to users. Providers are refining their platforms, making improvements that make research processes more user-friendly. The tools are robust and can streamline efficiency in many research areas, but a comprehensive platform still seems to be out of reach as no single solution is all-inclusive. While we are seeing evolution within qualtech, we are still not seeing revolution.

Technological advancements in the past few years alone have provided a plethora of solutions, but most are geared to remote and online qualitative research. The industry continues to question if and when in-person research will return. Facility-based research plummeted in 2020 due to stay-at-home orders, local mandates, and everyone’s hesitation to gather, and sadly continued to further decline in 2021. We aren’t seeing focus group facility research volume that matches levels of 2019 and years prior, and we may never see those same levels return, but in-person research is definitively increasing and 2022 is showing a strong rebound. Lenny Murphy of Greenbook shared that research has to be “Fit for purpose”. Charlie Rader of Procter & Gamble echoed this sentiment by sharing, “There’s only so much you can represent on screen”. In a nutshell, if it can be conducted remotely, then there is an abundance of technology that can support the chosen methodology; if however, your research requires use of senses (touch, taste, smell, etc.), it must be conducted face-to-face or via HUT.

Concerns with acquiring quality sample continues to plague researchers. On the quantitative side, bots and fraud are highly problematic. While the issue of fraud in research is not new, the global pandemic and corresponding increase in online research opportunities exacerbated an existing issue. Some brand researchers have claimed that they are removing up to 80% of their collected quantitative data due to confirmed or suspected fraudulent activity. L&E Research recently collaborated with CASE (Collaborate, Advocate, Spearhead, Educate) members to discuss some of the sample challenges researchers face in an in-depth webinar discussion. Qualitative research is much more involved for participants, so while qualitative still experiences fraud, an experienced qualitative research recruitment company can help to mitigate most of this by administering a high-quality and targeted screener to secure the right panelists.

On the qualitative side, the struggle seems to be focused on supply and engagement. Many research buyers have cited participant supply as another obstacle impeding research objectives. It’s no coincidence that the top complaint qualitative recruiters hear from potential panelists is along the lines of, “I complete all these screeners, but I’m never selected to participate in a research study!” As an industry, we must all take a strategic approach to building a more efficient process that keeps participants engaged in research; without their feedback and opinions, research will cease. They are the lifeblood of our industry, and unfortunately, we haven’t done our due diligence to assure a positive experience for them.

In order to keep qualitative panelists engaged, they need to qualify to participate in research, and not simply fill out screeners. Screeners must be developed to only include questions that would qualify or disqualify potential candidates, including necessary demographics. If a question is only collecting information, save it to ask in the research session with those who do meet the qualifications. Screeners should never be used to collect market data, and suppliers can reduce this practice by only providing screening data for those who are fully qualified and recruited for the study. As an industry, we need to collaboratively bid a fond farewell to 10-page / 30-question screeners – this taxes participants and is damaging to the overall participant ecosystem.

Another execrable practice observed is the use of quantitative panels for high-incidence qualitative recruitment due to the lower price point. While researchers can procure low-cost recruitment within quantitative panels, the research results may not elicit qualitative insights. Qualitative research recruiters must build their member databases with a focus on ID-validated and articulate participants who have demonstrated a willingness to spend more time on an involved activity: some people simply aren’t built for qualitative research. On the flip side, quantitative panels should be built for volume, and level of respondent engagement is less a factor since most obligations are brief and generally don’t require a great volume of elaborate open-ended answers. High-incidence studies are used by many qualitative recruitment suppliers to “reward” members who have completed a multitude of screeners, but never seem to fit the qualifying criteria. In order to contribute to the overarching health of the respondent pool that all qualitative recruitment suppliers share, if your research target is low incidence, or even specific product users, consider balancing your recruitment needs with a segment of high incidence recruits; product non-users may have an unique perspective to offer!

Part of ensuring respondent engagement is offering a hearty incentive amount. “Cash is King”, and that holds true when considering panelist incentives. A solid incentive not only encourages a strong response rate to screening outreach efforts (to provide a more diverse pool of candidates screened), but once secured, participants are more apt to complete the obligation to which they have committed. When multi-million/billion dollar decisions are on the line, and consumer insights are part of the decision-making process, is the combination of an uber-cheap recruitment fee and an inequitable incentive a risk you want to take if it means the participants are not of the highest caliber?

Finally, a powerful and dynamic technology suite to manage the database is critical for recruitment suppliers to operate efficiently. Software that can accurately track and retroactively update data points based on current screener information will reduce touchpoints to non-viable candidates, so they are completing fewer screeners, therefore eliminating frustration with our industry’s usual participant procurement process. Our industry has made strides when it comes to database technology, but we still have a long way to go. One day, perhaps recruitment can be conducted without the use of a screener, by using personas, or solely through technology, shifting away from the “ask” environment, and relying on observed data.

We hope you found this summary to be helpful! If you didn’t register for this webinar you can watch this webinar in its entirety by clicking here.

Be on the lookout for our next webinar, which will be in January 2023. If you can’t wait until then, you can always view our on-demand webinars. Don’t forget to join our mailing list so you can keep up with what is happening at L&E!

Digital qual – the 51st state?

DWG Admin on November 4, 2022

New developments in online qualitative research put larger groups of respondents at your fingertips, but quality is still paramount

In times of constant change – and the digital qualitative research world is certainly experiencing constant change – two things are vitally important.

It’s crucial to keep your eye on what is changing – the new opportunities, technologies and ideas that are springing up. It’s also crucial to keep a place in your thinking for what has not changed – the basic principles that consumer research rests on.

Fresh insight, representative recruitment, best operational practice, and data quality are no less important because methods are in flux. In fact, they’re more important. They’re the thread of continuity which lets you know you’re delivering the best insight and outcomes for the companies you work with.

If new methods can’t help you improve on these benchmarks, it’s not worth spending your valuable budget on experimenting with them. Fortunately, many new methods can, and there’s more activity at the cutting edge of online qualitative research than ever.

Before we dive into that activity it’s worth a recap of the last few years to understand why and how the landscape of digital qual is changing so much.

You’ll hear a lot of people say that COVID-19 changed everything. It would be truer to say that the Covid pandemic accelerated everything. The pandemic put a temporary halt to traditional face-to-face methods and forced a large-scale shift to online qualitative research.

However, the pressure to embrace digital qual existed well before 2019. The core functions of online insight platforms were already tried-and-tested by 2019, and machine learning and qual-at-scale were developing fast.

What the pandemic changed more than anything was a rethink of the old qualitative dogma that digital research could never deliver the nuance and depth of face-to-face work. As billions became more familiar and comfortable with video call technology it was obvious that remote video could actually deliver the kinds of deep, human communication great qualitative work relies on.

So as hybrid work comes to be a working norm, it’s no surprise that hybrid qual is, too. Projects and even individual groups mix video call technology with face-to-face research, just like hybrid meetings do.

The watchwords here are reliability and smoothness. You can’t let technical glitches or tricky user interfaces break the flow of your meeting. But platforms like Forsta, which specialize in this kind of hybrid online/face-to-face work also offer suites of analytics, editing and content management tools to draw the maximum insight from these groups.

Most of the innovation in digital qualitative research is happening on purely online platforms, though. The range of start-ups and new offers in the sector is huge and difficult to sift through. It’s all too easy to be wary of the wild claims being made. It can be useful to simplify things and think of digital qualitative platforms as offering three broad benefits.

They can save time, by automating or cutting out steps in the qual process that require a lot of work, like transcription, translation, curation, and editing.
They can produce deeper insights by helping users do parts of the process better. Often this involves machine learning or unstructured data analysis, or data integration with other non-qualitative insight sources like sales or CRM data.
Or they can create new insights by doing things it was simply not possible to do any more – this is what the “qual-at-scale” sector, which creates custom qual data using hundreds or thousands of participants rather than dozens, is promising. It’s also where the predictive end of machine learning and AI fits in.

If a new provider or platform can credibly promise one or more of these, they’re worth looking at.

Tools that automate research and save time are the least glamorous area but can often be the most useful. Platforms like Qualie specialize in large-scale video analysis, tagging and search, which have all improved hugely since the pandemic made front-facing video content the standard mode of online qual. Improvements are still happening – for instance algorithmic tools that can go beyond simply searching video data for relevant content but also edit it automatically to turn it into verbatim clips and quotes.

It’s now common for digital qual platforms to incorporate some level of machine learning and NLP (Natural Language Processing) to analyze data. New start-ups specializing in this continue to launch, but the improvements in technology here are mostly ones of degree. In other words, new tools are becoming better at understanding language, uncovering themes, automatically coding data, and other tasks – but these are incremental, not revolutionary leaps forward.

The thing to remember is that using AI as a qual researcher is always a kind of duet between you and the algorithm. AI is still only as good as the human collaborators who train it and interpret its findings. Curiosity and open-mindedness are still the friends of great qual research, and false certainty is its enemy, something technology will never change.

Finally, the most exciting work in the digital qualitative research space is still being done with qualitative research at scale with tools like Remesh, which allow mass qualitative studies to happen. Social media used to be described as a focus group with a million people, but as we now know, a million self-selecting people (half of whom are bots) may not be so useful. But a focus group of 1000 people, all able to respond and interact like a Twitch stream with analysis in real time – that’s a kind of insight platform researchers have never had access to before.

For that kind of rapid analysis of unstructured data to have value, you need to make sure your inputs are gold-standard, and your qualitative research recruitment is second to none. Data quality and recruitment expertise become more important than ever.

New tools and ideas are reshaping the sector all the time. What impact, for instance, will AI art have on qualitative NPD research, as participants can bring new ideas to visual life simply by describing them?

The pace of change keeps increasing, and we live in revolutionary times for our industry. But in revolutionary times, expertise, and a bedrock of quality matter more than ever.

The Latest in Qualitative Research Technology – Part 2

DWG Admin on December 7, 2021

New Tools for Better Insights

In part one of this blog, we discussed using AI to aggregate feedback and tap into data sources that may be too large or complex to analyze without AI. Yogi is emerging as a useful platform to identify and analyze data efficiently.

Yogi uses AI to aggregate, organize, and analyze data, which adds structure and meaning to reviews and ratings. Yogi power comes from combining consumer sentiment with topics of conversation to make it easy to identify likes, dislikes, competitors and challenges.

Live Conversations with 1,000 Participants

One of our newest partners, Remesh, allows customers to have a live conversation with audiences at a much larger scale, using AI to analyze and organize responses in real-time. In a live Remesh conversation, it’s possible to collect responses from up to 1,000 panelists in a single session.

The client develops the discussion guide for their exploratory qualitative research while L&E Research recruits the target audience to participate. The discussion guide can be customized using open-ended questions, polls, and even images or videos. The moderator selects pre-loaded questions from the discussion guide or can modify or write new questions in real time as the audience responses stream in, making the discussion fully customizable based on the responses received.

Quality Recruiting is Still Key

L&E Research taps into our panel of over one million US members to find the appropriate targets so customers can get the answers they need, and stakeholders are equipped to make decisions to solve their brand challenges.

Once the target audience has been secured, and the discussion guide has been developed, the Remesh session can be conducted. This is a fantastic way to brainstorm a topic without panelists influencing others opinions, so that a large variety of fresh perspectives are brought to the proverbial table for the client to review.

On the other hand, if idea sharing within the session is desired to stimulate ideas, that can be achieved – it’s all fully customizable! As new ideas are presented, and feedback is provided, the moderator can make alterations to the discussion and add or remove questions if changes are warranted. Remesh can collect quantitative and qualitative responses within a single session, cutting down on time and expenses when both need to be achieved. Due to its scalability, it can reduce moderation time as one session can collect responses from up to 1,000 panelists.

If further in-depth discussions are necessary to complete the qualitative analysis, individual or small group sessions can be facilitated through L&E Research’s Virtual Facility via a video conferencing platform. Finishing the research with a smaller scale discussion offers researchers the opportunity to answer any pending questions or tie up loose ends they may have.

Multiple Tech Solutions, One Vendor

This hypothetical, multi-phased approach to both identifying and solving a research problem using just three vendors could streamline and simplify a company’s process and save researcher time. As Yogi and Remesh are both partners of L&E Research, this could all be done on one invoice though L&E, saving the finance and procurement departments time and resources as well.

The Latest in Qualitative Research Technology – Part 1

DWG Admin on September 23, 2021

Covid-19 Changed the Way We Conducted Research

Covid-19 impacted the market research industry in a countless number of ways. L&E Research, like many other qualitative research firms that offer in-person facility solutions, saw a drastic decline in face-to-face research. While researchers were already exploring remote options including asynchronous activities like bulletin boards, shop-alongs, home-use tests, and diaries, these were never meant to replace the importance and value of having a conversation in a one-on-one or group setting with a live audience.

The pandemic led to an increase in conducting synchronous sessions via video-conferencing software, and an overall increase in asynchronous research. But what about the projects that still needed to be conducted face-to-face? Qualitative research firms that offer in-person facility solutions had to implement changes that would reduce risk for all parties involved. By making their facilities as safe as possible for staff, clients, and participants, in-person research became a viable option once again.

Technology made it possible to move research online in a meaningful way, conduct new types of remote research activities, and allowed us to find and utilize new sources of research data.

The Way We Shopped Allowed an Untapped Data Source to Emerge

Of course, market research wasn’t the only activity to see a shift from in-person to online. There was an incredible surge in online shopping over the course of the pandemic – a whopping 32.4% increase over the previous year (https://www.digitalcommerce360.com/article/us-ecommerce-sales/). As review platforms such as Yogi saw an increase in the volume of online reviews and ratings, brands began to realize the importance of this previously untapped data source. It contains a wealth of unbiased feedback that simply needs to be analyzed in order for valuable insights to be found.

As New Tools Were Needed to Analyze This Data – Enter AI

Artificial Intelligence (AI) is the next step in automation when working with large amounts of data. While AI does not replace the insights professional, AI organizes and analyzes data and sentiments, which makes teams more efficient by removing menial tasks, such as tagging, collating, and structuring data, from the process. With advancements in AI, researchers can analyze observational data, and not simply base their findings on the “ask environment” (i.e. polling, interviews, etc.).

AI is best utilized when it can support and enhance the role of the researcher, not take the researcher out of the picture. A focus group still needs a trained moderator, and AI cannot replace that. AI can help that moderator achieve greater things with the right technology.

Using Qualitative and Quantitative for Agile Innovation

Research has been moving towards a more inclusive hybrid approach, incorporating both quantitative and qualitative into the entire innovation cycle. Rather than using an anchored “waterfall” approach, it’s more comprehensive for decision making to now use an “agile” approach that is more frequent and lighter in nature and may include more tools along the way rather than one large strategy. This gives researchers a collection of different types of data to inform their decisions.

Using the combination of these technologies to fulfill research needs could be quite the powerhouse and can leverage both the “observe” and “ask” environments. For example, a company could use Yogi to analyze their reviews and ratings across a variety of sites to identify a problem they may have with a specific product, or perhaps it may reveal an issue with their brand perception of which they were unaware, or it could simply show them how their competitor may be outperforming them. Whatever the case may be, based on their analysis of existing reviews and ratings, using observational data, they have now identified a problem and can develop a purpose statement which will guide the research team on the work that must be accomplished throughout the research process.

Developments in technology go far beyond just helping researchers shift from in-person to remote. With changes in the macro environment like massive shifts to online shopping, researchers need to have the tools to identify new sources of data and analyze them efficiently. AI and other software will never replace the insights professional, but having the right technology can allow research to become faster, more agile and more innovative.