Is machine learning getting to grip with sarcasm? Sentiment and Semantics

It’s no secret that qualitative data is harder to analyze than quantitative. The replacement of ticked boxes for a plethora of data from verbatim, videos, group conversations and observations means that participant answers are vastly more time consuming to categorize. Not only this, but the context and motivations of the individuals need to be considered in addition to their responses and behaviors. But the effort is worth it; you can work to really understand consumer behavior and get to the ‘why’ behind the ‘what’.

Of the many techniques used to analyze qualitative information, sentiment analysis and semantic analysis are two we often come across. These provide insight into the data by looking at attitudes and themes. And as new technology continues to make its way into qualitative analysis, we need to understand how it can impact these two methods of analysis, and how it can help develop more meaningful insight.

As a reminder, sentiment analysis categorizes language based on opinions, often checking for a positive or negative viewpoint. This can include, for example, the views of consumers expressed on social media or online communities toward a specific product, idea or service. Although this deep analysis can be done by humans, it is quicker using technology – most often machine learning. Machine learning is helpful in analyzing large quantities of information and making judgements based on language that would be far too time consuming for researchers.

However, it’s important that sentiment analysis is not entirely reliant on technology. We are often asked to analyze data in a binary way, but the expressed sentiment is rarely that black and white. Technology often makes assumptions about the sentiment of a response to make it fit a binary format, while humans would be more cautious to force content into a category. Moreover, while humans can contextualize written language and understand colloquialisms and contradictions, as well as turns of phrase and humor such as sarcasm, machines struggle with this. Machines simply cannot understand all the nuances of language and its grammatical patterns to the same accuracy that a human can. In spite of the claims of some of the technology providers, we are far off being in a place where text can be analyzed by a machine as well as by a human. It is therefore important to use a joint approach to sentiment analysis; machines may make things quicker, but they’re also more likely to create inaccurate data through generalization, lack of contextual knowledge and a lack of natural language skills.

Semantic analysis, on the other hand, generates meaning by grouping together the themes in qualitative data. It is often referred to as topic or keyword analysis. This is an important approach when trying to understand the important subjects in the field of study, regardless of how a participant feels about them. This is often useful in the early stages of a research project, when exploring a broad idea and looking into other relevant and more specific ideas that need to be examined further. As with sentiment analysis, this can easily be done by a computer, and indeed, it is far more efficient to do so, leaving the researcher with more time for deeper analytical thinking. Furthermore, a machine learning process can take place with no prior knowledge of the documents – again saving time. However, there are limitations, as the grouping of themes needs to go beyond basic classification to give a meaningful representation of the qualitative information.

So machine learning speeds up sentiment and semantic analysis; for text based analyses this can open up a whole new world of information about consumers from social media, communities, and other online resources. The tech guys are working hard to solve the nuances of language such as sarcasm – machine learning’s biggest barrier in sentiment analysis. And then with semantic analysis comes the challenge of context – such as the ability to distinguish Mars the chocolate bar from Mars the planet. Technology is certainly getting closer to solving these problems, but in the meantime, it’s nice to keep the humans around.

For more on qualitative analysis, download our white paper here.

Renee’s Takeaways – Get the scoop on L&E’s Qual Research Recruiting Webinar!

6 Steps to Perfect Qual Webinar Series Summary

Step 2 – Qualitative Research Recruiting Webinar – Webinar held on April 11, 2018

Let’s add another webinar filled with great information to the books!  In this webinar in our series, we focused on Qualitative Recruitment and got to spend some time with three experts in our industry.  First up, Teressa Johnnson, Director, Sensory Science and Technical Support at The Wendy’s Company.  Next, Heather Hall, Vice President of Project Management at Illumination Research.  And last but not least, our very own Brett Watkins, CEO and President of L&E Research.

In this webinar, we focused on Q&A based around Qualitative Recruitment, tips to help your recruiting go as smoothly as possible, and also help ward off any issues before they become issues starting with the design process.

Here’s what our panelists had to say!

1. We started by asking about kicking off a new project with a recruiting partner.  What might our panelists be able to share with an amateur researcher in regards to getting recruiting started off on the right foot?

  • Partner, partner, partner! You want to be sure to work with recruiting companies who not only care about helping YOU be successful, but who also align with your core values. In any good relationship, personal or business, communication and honesty are key elements to be able to have a true partnership.   In the end, you’ll want to be sure that you can trust your partner to do what is best for your project.
  • Be clear about your expectations for the recruit right from the start! Have a discussion about who your ideal respondent is. Ask your recruiting partner if they have any past experience with the population you are trying to reach, and if they can offer and advice on the best ways to reach them.
  • Ask your recruiting partner what their plan is to find your population. If plan A does not work, what’s plan B?  Having this mapped out at the beginning, will help keep your recruiting from getting hung up, and keep the process moving along.

2. Next we discussed what might be at the top of our panelist’s checklist when kicking off a new project.

  • Be sure that your recruiting partner is fully aligned with you! Make sure you are crystal clear on who you really want for your research.  This will help eliminate issues down the line.
  • Have a kick-off call! Extremely important to have a kick-off call with your recruiting partner before getting started.  In the age of email, some things may be uncovered in a phone call, that was not communicated in the email thread.  This is a good time for both parties to ask questions, gain clarification, brainstorm, etc.

3. We also touched a little bit on screener development.  What do our panelists always work into their screeners?

  • False, or catch questions. Ask about product usage, and include a few items in a list that do not actually exist.  If a potential respondent answers yes to using any of the false products, TERMINATE!
  • Include your plan B right in the screener! If you anticipate that a question may not work, or may be a sticking point, make your plan B known from the start as to not hold up recruiting.
  • Be careful of the way your questions are worded. It’s easy for a question to be interpreted by different people in different ways.  It’s extremely important to be sure that your questions do not have multiple ways they can be interpreted.  Let someone else on your team put their eyes on it, even take it for a test run if time allows.

4. Touching on recruiting partnerships again, we asked our panelists what some of their must haves are in a recruiting partner.

  • Communication is key. Over communication is even better!  Keeping the client in the loop, and always informed about what is going on with their recruit.  If they are anticipating issues, let the client know, and even come to the table with some suggestions on what can be done to keep the recruit moving right along.
  • Timely updates. A partner who understands that no news makes the client anxious!  All ties in with the above, and if there is an issue, knowing about is sooner than later is best to be able to stay on top of it.

5. Along the same vein, we asked our panelists what the best service is that their recruiting partner can provide.

  • Once a project is kicked off, get started quickly! If recruiting gets started quickly, you can have a gauge early on how the recruiting is going, and be able to identify any issues sooner rather than later.
  • Be honest about feasibility! Work with a partner who does their homework up front, and can let you know, with certainty, if they can recruit the population you are after. If they cannot, they should come to the table with what they CAN do.
  • A solid database. It’s helpful if a recruiting partner’s database holds a lot of preliminary information about potential respondents.  This way, you can focus on the meat of the recruit, and get the demographics and such out of the way up front.

6. One question I think we all ask now and then is how we avoid some of the common issues that may arise with recruiting.  Is there anything that can be done during the design process in order to get the most authentic participants?

  • Blind recruit. Set up your screener to hide the topic, but as mentioned earlier on, include the red herring questions.
  • Re-screen on-site. If someone was not truthful during the screening process, they will likely not remember how they answered if screened again.
  • Work with a recruiting partner who uses technology to track things such as past participation, participant behavior, etc., so you are able to weed out non-qualifying participants early on in the process. Also a partner who is always working to grow their database to ensure fresh participants.

7. New technology and recruiting.  What do our panelists think about it, what do they like, and what are some things that need to be overcome when working with it?

  • Online research. You can reach a lot of people who you may not have been able to reach before, expand your reach. One of the issues with this are bots, and how they can mimic human behavior to answer questions.  Better technology could be put in place to help with that.
  • Webcams can give us some great insights into consumer behavior, and that can work alone as the sole methodology. It’s even better when you can use it prior to groups or interviews to get the conversation started before the respondents even arrive on-site.

We hope you found this summary to be helpful! If you didn’t register for this webinar you can listen to this webinar in its entirety, or download a free, transcribed version by clicking here.

Be on the lookout for our next webinar, which will be this summer. If you can’t wait until then, you can always view our on-demand webinars and learn about the latest technology solutions. Don’t forget to subscribe to our blog so you can keep up with what is happening at L&E!

Until next time!

Your Research Design Engineer at L&E,

Renee Wyckoff

Webinar Recording | 6 Steps to Perfect Qual- Step 2: Qualitative Research Recruitment

6 Steps to Perfect Qual – Free L&E Webinar Series

On April 11, 2018, L&E hosted our second webinar in our “Six Steps to Perfect Qual” webinar series for 2018 with Teressa Johnson, Heather Hall and Brett Watkins as our panelists. The second step in our series focused on qualitative research recruiting and its importance to great qualitative research.

Miss out on this webinar? Not to fear, you can find the on-demand webinar recording here.

And if you haven’t read our next 2 white papers in the 6 step series, you can find those here.

B2B Qual – going beyond in-depth interviews

For the deepest, most comprehensive insights and personal opinions, it is difficult to beat the one-on-one interview.

However, there are other options.

Many people assume that B2B qualitative research cannot use the same variety of methods as B2C qual, or even B2B quant for that matter.

While it’s true that the in-depth interview is a great way of gaining deep insights into a participant’s thoughts, the vast potential of the internet means there are more convenient ways to reach professionals with tight schedules across varying locations.

Moreover, observational methods such as ethnography provide objective viewpoints of businesses that personal interviews may miss.

Online Focus Groups For B2B Qualitative Research

Admittedly, focus groups are less frequently used in B2B qualitative research than they are in B2C because of the time and geographical restrictions of participants.

B2B projects often require researchers to reach senior staff members to provide insights on specific issues and these are, of course, the people with the busiest schedules who have little spare time to provide feedback.

Moreover, geographical issues mean that even if spare time could be aligned between participants, there would be no convenient place for everyone to meet.

This issue can be solved by holding online focus groups so participants can log in to a virtual research space wherever they are.

And, to make sure participants join, remember to send a meeting request to participants’ work calendars so the event is firmly scheduled.

Flexible Online Discussion Groups For B2B Qualitative Research

Even better, for those with extremely tight schedules, is holding an online discussion group, allowing participants from any location to add to the conversation at a time that suits them.

This flexibility is important when respondents have little time to spare.

It will allow them to give more thoughtful, meaningful responses without the pressures of time and what is more it means they can talk with each other in a secure and controlled environment.

In an increasingly technological world, respondents often feel more comfortable giving their opinions to a faceless researcher online than they would to a real person who they may feel judged by.

Another benefit to researchers is that participants who do not engage satisfactorily in the discussion can be replaced, ensuring that productive discussion takes place between the moderators and participants (and the participants and participants) so authentic insight can be gathered.

This is also a great method for categorizing participants, as researchers can target specific group members with different questions to develop insight further.

B2B Ethnography: An Objective Approach To Qualitative Research

Depending on your research problem, B2B ethnography is also a great method to consider.

Although there can be a process involved in recruiting participants, a research technique that simply requires researchers to watch and learn is a great solution for participants with limited time as no changes to their normal habits are required.

This technique is excellent for research problems that look to assess behavior in the workplace and to observe the intricate happenings within a business, without asking employees for their own personal opinions which may be wound up in internal politics.

Ethnography is therefore a good way to get an objective B2B viewpoint.

Choosing The Right B2B Qualitative Research Methods

So, although interviews are important in B2B qualitative research, there are other, sometimes overlooked, methods which are a great fit for the B2B sector.

We all know that working life is busy, especially if you’re the CEO, so resources like online tools are useful for bringing people together regardless of location.

Online focus groups provide a universal research location, while online discussion groups allow research on demand for those whose unpredictable schedules mean they can’t commit to a specific time slot.

And for objective, unopinionated research, why not try ethnography, to see the background of a business without working around the time restraints of employees.

To access more information on qualitative data collection tools, download our white paper on fieldwork and data collection in qualitative research here.

Renee’s Takeaways – Quarterly Roundup

  1. Methodologies – There is no one-size fits all when it comes to methodology for your research. While focus groups definitely have their place and are necessary, we’re seeing more and more companies using behavioral methodologies such as in-home interviews, or video diaries, etc. to be able to capture those “in the moment” elements that you may not catch otherwise. People tend to act and respond more naturally in their own environment.
  1. Qualitative recruiting – There are steps YOU can take as a researcher to improve the quality of recruiting coming from your partners! It all starts with project design.  Clearly defined research questions, and a clear expectation of what you are hoping to find out from the data that is collected.  This will lead to clearly defined specifications as far as “who” you would like to speak to, and will provide the basis of the questions you will ask in your screener. The more heavy lifting done on the front end, the more it will help ensure truly qualified respondents on the back end.
  1. The future of MR data collection/analysis – Is the future of MR data collection and analysis all about quick and dirty? Yes, to an extent. While it seems that clients are looking to do the research faster, and get the results faster, when it comes to the analysis of the data collected, the human element is still very important in order to be most accurate.  There are ranges of emotion and detail that software platforms are not able to detect and accurately identify.   If that is understood, and taken into consideration, then a fully automated process could work. Otherwise, people power is still needed!
  1. Screener development – As we already know, a streamlined, well written screener is going to produce better respondents! A few things I’ve learned to be effective along the way are the following –
    • Keep it short and sweet! The main goal here is to be sure that the person qualifies for the upcoming research, not to find out all of the information you need to know in advance of the research. Use the screener for the important qualifying questions, dig deeper during your research.
    • Try to ask easy questions that are likely to disqualify early on before getting to the meat. This will help the recruiting process go faster.
    • If there is a typing tool/algorithm that goes along with your screener, suggest to your recruiting partner that they run potential respondents through the tool first. If they are not the desired segment, or one of the desired segments, why continue? Just another way to expedite the process.
  1. A subject not related to MR at all! – Here at L&E, we have an internal learning system which is meant for both professional and personal enrichment alike. When I come to a module that I particularly enjoy, I like to share it! This particular module was about happiness.  It is largely thought that success will lead to happiness, when it has been studied that it is actually the other way around.  Happy people are more productive, more creative, and able to have richer, more meaningful relationships! These things are successes in and of themselves, and there are several small things one can do to inspire happiness in themselves and in others.  Just a few of my favorites include the following – 1) Express gratitude. Take a few minutes each day to think about a few things you are grateful for.  2) Practice random acts of kindness.  Do something nice for someone, it makes them feel good, and you feel even better! 3) Appreciate the little things. Don’t forget to stop and smell the roses!

Until next time!

Your Research Design Engineer at L&E,

Renee Wyckoff

Video open-ends – think outside the box

Consumers have a lot of opinions and valuable advice to offer, but how can we ensure they’re heard?

Video puts data collection tools right into the hands of consumers. It does a great job in engaging research participants so that they know they are really being listened to. It puts them center stage, making them feel their worth and their value. With video open ends, participants aren’t constrained by text box responses, or even worse, a pre-written answer key. They are able to elaborate on their thoughts and opinions, while conveying their true emotions and feelings. And this is all done much quicker and easier, and on a more personal level, than typing in a box.

For researchers, video analysis means accessing deep insight from responses that are much richer than reading numbers or text alone. In fact, text box open-ends on average only get three or four word responses. While video gives researchers access to six times more content.

But although video brings a lot to the table, there’s still some confusion around how researchers can fully harness the value of video open ends.

Open-ended video questions can seem out of reach, especially for independent research consultants. But as a powerful source of insight, they’re well worth it. After all, it’s our jobs as researchers to take advantage of every technology and every resource available so as to access the raw emotions and the complex opinions of our participants, to elicit deeper insight to share with our clients.

So, how can you make video work for your research project?

Add video to current methodologies. Video works very well with in the world of qual, but it also applies to quant projects. Video helps to marry the two for richer insight, to let consumers tell the story behind the data. Video can be added to online surveys, NPS studies, online communities, brand trackers, ad testing, focus groups, ethnographies and more.

Take advantage of mobile. Mobile phones and laptops can easily capture video from anywhere in the world, at any time. Mobile is giving consumers 24/7 access to sharing their views with brands. Researchers and brands can reach out to consumers at any time with ‘push’ content to solicit video feedback in the moment.

Show and tell. Video is a great way to put products in front of participants. It can help the participants understand how a new product is used in a certain setting, and how it will be positioned, in order to find out how the messaging and language will resonate with target consumers.

Put it into context. Video should be used to go into greater depth, to get the fuller story, and to put everything into context for the researcher. A video can show the researcher how the participant uses the product in their home, or what products fill the shelves at the store.

Feelings. The facial expressions and the pure emotion that is picked up in a video within ten seconds can say much more than the typical text box response of 50 characters. Make sure the questions are designed to explore the participant’s feelings and emotions, and to probe their opinions.

Don’t settle on quality. The quality of the video software used is important as it can affect both the participant’s experience and the insight collected. Poor quality video can negatively impact the quality of the insight, so if you can control the video quality, so if you are using the software in a survey for example, the video must be of a standard that doesn’t distract or negatively influence the participants.

It’s easy to dismiss the video trend in research as a tool that is out of reach and too complex to integrate into current methodologies, but there are a lot of partners and tools out there to help. If you’re looking for more on video and other data collection tools, download our white paper on fieldwork and data collection in qualitative research here.

6 Steps to Perfect Qual Series: Step 3 – Fieldwork & Data Collection | New White Paper from L&E

6 Steps to Perfect Qual

White Paper Download

Step 3: Fieldwork & Data Collection

When you back your fieldwork and data collection with the latest technology, you can’t go wrong.

Download the latest L&E trust guide to access new (and go-to) qualitative research techniques, along with the in-the-moment tools, video analytics, and other technology they run on. In this paper we’ll also share practical tips in choosing the right facility and creating an optimal research environment for participants.

This paper includes:

  • Emerging fieldwork techniques (and the technology they run on)
  • Practical tips on making your participants and clients comfortable on site
  • How to deliver an environment that will optimize data collection
  • Strategy for choosing the right facility

To continue reading, download our latest white paper by clicking here (no form to fill out).

More time to focus on the good (participants)

Looking for respondents who wear glasses three times a week, eat steak tartare once a month, and go for just one run a year (every January)?

In our last blog, we discussed the power in using recruitment software to fend off and quarantine problem participants, such as those troublesome ‘professionals’. The latest technology and quality recruitment methods take care of these problem participants for us – allowing us to move on and focus on the good.

With technology in defense mode, recruiting companies can switch their focus to offense -to proactively search for the best participants – those ideal candidates most suited for any qualitative research project, no matter how targeted.

So, how do you know if a participant is the right participant? Of course, you want to attract only the people that are in research for the right reasons. Researchers want the people who are excited to help and want to share their opinions about the products and services they are passionate about. They want to help a brand meet their needs.

But once again, recruitment technology and leading software platforms are here to support an easy and high-quality recruitment process. The technology secures the highest quality participants by tracking behavior, while also providing a great way to engage with participants prior to the in-person research taking place.

So how does a research company locate these quality recruits?

The technology makes it easy to search for top recruits. It uses specific keywords and product mentions to perform a thorough search, in order to continue building the database. The software finds and adds more than 100 new participants each day. The search criteria and strategy in itself is rigorous, and it includes filtering tools from basic demographics to many other data points (over a hundred to be exact). For example, basic data points would include age, ethnicity and marital status. More specific criteria could be home type (such as owning a town home), food restrictions (gluten free) and technology (owning a Google Home). This allows researchers to zero in on precise recruits. So basically, you can afford to be picky.

In order to keep things accurate, the management tools also provide respondents with 24/7 access to their profiles for continual updates. The participants are also constantly tracked. For example, tracking participation and behavior in previous studies can eliminate respondents based on participation requirements. Behavior tracking also allows panel companies to build relationships and tap into new consumer segments. Reward programs make a big difference, and they can really work in terms of referrals. The software runs reward programs that help to build panel through the company’s funded initiatives and people who are dedicated to panel growth and panel integrity.

So, you might be wondering if the technology can be used to help you find an entrepreneur in Arthur, Nebraska (population 119)? The answer is yes.

Looking for more best practices on recruitment? Download our latest white paper on best practices in qualitative research recruitment here (no form to fill out).

The myth of cheaters and repeaters – An insider’s look into qualitative research participants

If you think professional respondents and problem participants are disrupting the quality of your insight, then you’ve fallen for the tired propaganda of those on a mission to steer clients away from qualitative research.

That’s right, it’s a myth.

Of course, professional participants do exist, and a lot of people only want to help for the sake of the incentive. But the thing is, they don’t actually prove a problem for qualitative researchers. It’s not because these people don’t try to cause a problem, but it’s because as an industry we have barricades in place to fend them off completely. And if they slip through, we have the knowhow to quarantine them before they can affect the insight. Therefore, although cheaters and repeaters do exist, they just don’t make it in the world of qual – and certainly not in our panels.

The first thing to say is that it’s not proprietary screening processes, cold calling or digital marketing alone that prevents these naughty participants from passing through. It’s technology that does the trick.

It’s all about the pre-emptive maneuvers. Having the right technology in place to support the recruitment process is standard for the respected qualitative panel companies. Sophisticated database management systems diligently track behavior, and therefore they can denote the problem participants while making them unaware of their disqualification.

Of course, there are the lower quality panel companies among us who simply don’t invest in the proper panel development. They’re the companies who are busy trying to convince clients of their ‘secret sauce’, which is said to defend against professional participants. But that sauce just doesn’t exist. It takes the right technology, management systems, and human engagement to immediately disqualify the people who are just in it for the money.

The software used by the well-respected recruitment companies tracks participation behavior on an ongoing basis, and quickly identifies any issues from the screener questions at the very start. This allows us to rule out or eliminate the problem participants before they can influence the insight. In fact, the technology can track over a hundred different data points for each respondent to ensure the participants are of the highest quality. With these intricate and all-encompassing searches, the system is able to find 120-130 new participants every day to join our already large panel of extremely engaged people, which means we can avoid recycling the same participants over and over again. This active search strategy offers plenty of first-timers who are prioritized for lists, which eliminates professionals. But of course, the search and screening processes run by the software is always followed by real human interaction – a final opportunity to verify the quality of the participant. At L&E, we have someone on staff whose full-time job is dedicated to building the quality of our database, so problem participants don’t even have a chance.

So, it’s not a ‘secret sauce’, but a rigorous process of selection and elimination, powered by state-of-the-art database software, supported by the latest technology, and backed with real human engagement.

Plus, it’s that same technology that supports our effort in finding the very best respondents for each individual qualitative project. While fighting off the naughty guys, the software provides a great way to engage with the good guys, which helps to build relationships and encourage participation. Stay tuned for more on how technology helps us engage with only the best participants in our next blog.

And in the meantime, you can download our latest white paper on best practices in qualitative research recruitment here (no form to fill out).

The world’s largest library – Putting perspective into research design

The research design stage of a qualitative project often revolves around the research question. In fact, the key is to come up with a research design that will guarantee an answer to that question. The problem is that in doing so, we often forget to take in the bigger picture because our focus is too narrow.

Over time we often become ingrained in our approaches and forget the wealth of information out there, or even right under our noses. As students we are adept at finding (often free) resources, but often as we progress in our careers we forget those useful sites and sources.

But many of the answers we seek as researchers are already there – they are just hidden within existing research readily available from the world’s largest library (the web). Desk research puts your project into a broader context in order to guide the research design, and although it may seem boring, mundane and tedious, it’s an important, practical, and very valuable tool for researchers. And best of all? It’s free and readily available.

You don’t have to reinvent the wheel

Every year, there are millions of studies completed and they often reoccur to keep the research up to date. It might take a little bit of time to sort through the information, and you might need to rework some of it to get the answer you’re looking for, but if you spend the time doing the desk research, you could be saving a lot of time and money for both yourself and your client by avoiding repeat questions for information that is already out there. Likewise, if you don’t examine previous research, how will you know if you’re actually going to find something new for your client?

Mix in some big data

Doing some desk research can offer valuable information for next to nothing. At L&E, we’re all about the word of qual, but we also know that the most respected research projects cross borders from qual into quant using a mixed methodology approach. For small qualitative consultants this can be difficult due to strict project budgets. But luckily, the internet is the largest, free source of statistics and data relevant to any industry, any subject, and any project.

Find the bigger picture

Existing research can add value to qualitative research results by putting the findings into a broader context. Whether it’s comparing your results across different industries, or even competitors, it’s worth doing the background work.

Here are some useful resources for desk research.

And here are some tips for your search.

  • Consider word choice. Could a different word, or arrangement of words, help refine your results?
  • Consider different topics. Could searching for similar topics help find more information relating to your research question?
  • Consider the source. As a researcher, question how reliable/accurate/relevant the findings are.

There are many reasons to take advantage of existing data. A little bit of extra time spent on desk research in the design stage will save valuable time later on, allowing greater time to be spent on analysis of data. If you’d like to know more about the design stage, including the client problem, the research question, choosing the right methodology, tips for moderators, and new technology to support the design, check out our white paper on Qualitative Research Design here.