Types of User Research Methods

In the previous chapters, we’ve covered the fundamentals of UX research, gone over how to create a user research plan, and went over a foolproof framework for user research recruiting.

At this point, you’re probably brimming with all the essential knowledge and feeling ready to actually, y’know, do some user research.

But before we get into the individual methods (which we will, in depth), it’s worth taking time to review the different types of UX research methods at your disposal. Because there are a lot of different types of user research. 

Many methods are best suited to a particular scenario. Some only work well when combined with other, specifically complementary methods. And almost all deliver more valuable results that can be used to help groups across the organization—from marketing, customer experience, product, and design teams—make more informed decisions when they are part of a comprehensive hybrid research strategy.

That’s why, before you dive into the trenches, it’s helpful to take a step back and look at some of the broader categories that define different types of user research. 

This chapter provides the 30,000-foot view that gives you the overall lay of the land complete with the terms and topics that will help you decide which research method is best for your project. 

Let’s get to it.

In this chapter:

• Qualitative vs. quantitative methods
• Generative vs. evaluative methods
• Attitudinal vs. behavioral methods
• Remote vs. in-person user research 
• Moderated vs. unmoderated methods
• Which user research method should you use?

Qualitative vs. quantitative user research methods

The primary differences between qualitative and quantitative UX research have to do with how the data is collected and the nature of the data itself. 

Qualitative UX research typically involves collecting data through direct observation of a small group of people in order to assess behavior and answer the question: “Why?” Common qualitative research methods include interviews, focus groups, field studies (and other ethnographic studies), diary studies, usability tests, and co-design sessions that generate qualitative insights into user behavior.

Quantitative UX research, on the other hand, usually involves collecting data from a much larger group of people in order to quantify a problem by answering the questions, “How much?” and “How many?” Common quantitative research methods include usability studies, user surveys, click tests, card sorts, and A/B tests.

Which type of method you choose depends on the kind of question you’re asking. The qualitative insights and quantitative data produced by each type of research have unique strengths and weaknesses. 

Quantitative data provides very clear and unambiguous information about variables like how much, how many, and how often. The numerical nature of the data makes it easier to analyze, but it can also lack enough context to make informed decisions. For example, while quantitative research can tell you fairly quickly and easily how often a set of users performs a particular task, that information is only valuable if you also have the context to judge whether that rate of task performance is good or bad, which could have implications for your user experience design. 

Qualitative data is more challenging to analyze and interpret. It is presented in the form of unstructured or semi-structured observational findings like comments, preferences, and motivations. While this kind of data usually contains the context, it doesn’t offer a black-and-white output, relying instead on researcher interpretation. It does, however, help identify root causes of behaviors from real users, which then makes it easier to develop appropriate solutions. 

The two types of data may seem worlds apart, but they are actually highly complementary. When in doubt, mix methods for a more holistic understanding of users and a more complete view of the problem you’re trying to solve.

 As for whether it’s a good idea to skip the qualitative bit and lean into the statistics, well, Jakob Nielsen of Nielsen Norman Group had a few things to say about that:

“Quantitative studies must be done exactly right in every detail or the numbers will be deceptive. There are so many pitfalls that you're likely to land in one of them and get into trouble.

If you rely on numbers without insights, you don't have backup when things go wrong. You'll stumble down the wrong path, because that's where the numbers will lead.

Qualitative studies are less brittle and thus less likely to break under the strain of a few methodological weaknesses. Even if your study isn't perfect in every last detail, you'll still get mostly good results from a qualitative method that relies on understanding users and their observed behavior.”

Pro Tip: User Interviews is the fastest way to recruit participants for any kind of research and user testing. Talk to sales or sign up for a free account today.

Types of qualitative research 

The various qualitative research methods can be further categorized into 5 main research types: 

1. Ethnographic studies—observation of study participants in their natural environment to gain a better understanding of context, cultural insights, and participant’s day-to-day lives.
2. Narrative—in-depth research (typically interviews) with a small number of target users to uncover narrative themes and actionable insights that illustrate how various factors can affect events and relationships.
3. Phenomenological—a variety of tactics (interviews, observation, documentation, etc.) used to describe and interpret a specific phenomenon or event, including participant perceptions and motivations.
4. Grounded theory—long-term, in-depth research to discover the social and psychological processes behind an event or situation, including its causes.
5. Case studies—a detailed accounting, either explanatory or exploratory, of an individual, organization, or event that reflects how things happen in the real world.

Note: That’s a good, concise overview of the differences between qualitative and quantitative research. If you’re looking for a deeper dive into the whole quant/qual thing, never fear—we’ve written a whole chapter on the subject.

Generative vs. evaluative user research methods

The primary differences between generative and evaluative UX research have to do with when the data is collected and why. Generative methods help you identify opportunities and ideas, while evaluative or evaluation research methods help you figure out if your existing solution is on the right track. 

Generative research (sometimes called foundational, exploratory, or (as in this Field Guide) discovery research) helps researchers gain a deep and highly detailed understanding of the target audience, the market, and even internal project goals. It does a lot more than simply describe a user persona—it gets into the nitty-gritty of the real-world person who may eventually become a user. Generative research uses direct observation, deep inquiry, and careful analysis to develop a fully rounded, 360-degree understanding of the human beings in question—who they are, what their experiences are like (in relation to a product throughout their user journeys and to life in general), what they care about, what they believe in, how they think about the world, what drives their behavior and decisions.

The goal of discovery research is to unearth opportunities to innovate new solutions that will meet a specific and real need in the market. Without this deep understanding of users, it’s all too easy to head off and develop a product no one wants. In short, this type of research can be highly impactful on the product design process. 

Evaluative research is used to evaluate people’s responses to a product or solution. These user research techniques come into play beyond the first stages of product development, but not that much later. Evaluative methods can deliver valuable insights as soon as you have an initial concept (even just a rough sketch or representative prototype of a product design), and should be used throughout the design and development process as a kind of reality check. Continuously putting design iterations in front of your target audience for user feedback helps ensure that the final product delivers the experience people want while providing the solution they need.

Used together, generative and evaluative methods are like two sides of the same coin. One side helps you identify and define the problem you need to solve, while the other side makes sure you’re building the right solution to meet the needs of potential users.

Discovery (generative) methods

Generative research methods include things like stakeholder interviews to uncover internal goals, field studies to provide real-life context, and in-depth user interviews that ask open-ended questions.

Each of these methods uses a combination of first-hand observation, deep inquiry, and data analysis to ensure you’re building the right product at the right time for the right audience. Generative research methods often allow researchers to observe every aspect of a study participant’s engagement—verbal responses, actions, behavior, body language, and so forth. Researchers can gain a deeper understanding of the preferences and mental models by asking follow-up questions to uncover the “why” behind certain choices and behaviors. Any data—whether qualitative insights or quantitative insights—is analyzed to reveal relevant patterns in the responses. 

📖 Read more about Discovery Research Methods

Evaluative methods

There are a wide range of evaluative research methods, many of which are most valuable at specific stages of product development. Their purpose is to validate whether or not your in-progress design is effectively solving the problem you identified during your generative research.

Evaluative methods employ many different approaches—from tree testing to qualitative usability testing, accessibility testing, A/B testing, and more—to determine what is and isn’t working with your designs and design process, learnings that can be shared with user experience designers and broader design teams.  

📖 Read more about Evaluative Research Methods

Continuous research methods

Research doesn’t end once you’ve launched. In fact, it’s incredibly important to engage in ongoing listening and dialog with real users using methods like NPS and other user feedback surveys, product analytics, as well as customer support and feedback channels. Keeping the lines open is a critical part of sustaining your product’s value to your target users over the long term. 

Not only do continuous research methods give you your first look at how people experience your product in real life, they also help you adapt and evolve as things change. (And things will change—maintaining an ongoing conversation with real users and really listening will help ensure that you don’t get left behind.)

📖 Read more about Continuous Research Methods

Attitudinal vs. behavioral user research methods

We know it’s shocking, but people often say one thing and do another. The disparity is usually unintentional, which is one of the reasons it’s so important to use UX research methods that collect both behavioral and attitudinal data. 

Attitudinal research methods rely on self-reported data. This means that they reflect people’s stated beliefs, perceptions, and expectations. This category includes things like interviews, user surveys, focus groups, and card sorts—all of which ask study participants to tell researchers what they think.

The user feedback and insights from attitudinal research methods are valuable, but they also need to be taken with a grain of salt. People sometimes fall short of telling you what they really think or are unable to fully articulate their perceptions. And even when they think they are telling you the whole truth and nothing but the truth, it can turn out that their expectations don’t match up with reality. (This is because human beings are notoriously bad at accurately predicting their own behavior.)

Despite these cautions, attitudinal research methods can still be helpful as a way of uncovering a participant’s mental model, which can then be used to help user experience designers shape their designs to better meet user expectations.

Behavioral research methods are based on direct observation as a study participant interacts with a prototype of a product design or a finished product. Researchers often prefer these user testing methods since they are able to deliver more reliable, actionable insights and a deep understanding of users based on real-world scenarios. Research methods that study a user’s actual actions throughout their user journeys include things like eye tracking, A/B tests, tree tests, first-click tests, and also user analytics.

There are also a number of user research methods and user research techniques—user interviews and task analysis, for example—that can produce either attitudinal or behavioral data from potential users. In any situation, best practices include collecting a variety of both attitudinal and behavioral data.

Remote vs. in-person user research 

The biggest and most obvious difference between in-person and remote research is that with remote research, the participant and the researcher are in different locations. 

Here is a quick run down of some of the advantages most often associated with each approach:

In-person research:

• Is easier to moderate because participants tend to be more attentive and can ask questions if they get stuck.
• Offer a more complete view of participant responses because researchers can combine verbal response and task-related actions with non-verbal cues.
• Provides more opportunities to ask follow-up questions or make additional in-the-moment inquiries about participant choices or thought processes.
• Gives researchers more control over the environment, including types of devices and software being used, internet connection, data security, noise, interruptions, and so forth.
• Creates better group dynamics, which can contribute to idea generation and useful dialog depending on the type of research being performed.
  

Remote research:

• Allows for much greater scheduling flexibility since there is no travel required.
• Is less expensive because there are no space-related costs and both recruitment and compensation costs are often lower than for in-person studies.
• Makes it easier to recruit a larger, more diverse group of participants.
• Comes with fewer recruitment constraints such as availability or geographic location.
• Can provide a more accurate reflection of real-life usage because participants can use their own equipment.
  

Remote research isn’t suited for all research methods. For example, ethnographic field studies are particularly hard to replicate remotely. 

But in many cases, the benefits of in-person research can be replicated with remote user research techniques, giving researchers the best of both worlds: ultimate flexibility and cost-effectiveness without sacrificing the quality of responses.

Moderated vs. unmoderated user research methods

The difference between moderated and unmoderated research methods is the role of the researcher. 

Unmoderated or automated research involves unobserved tests that a participant can complete at their own pace. The participant interacts with the product via an online platform or software that prompts them to answer specific questions or perform specific tasks. 

The nature of this kind of test makes it easier, less expensive, and faster to run than research facilitated by a professional moderator. The data from unmoderated research studies tends to offer quantitative insights rather than qualitative insights into user behaviors, although some unmoderated tests include the option for participants to provide a recorded self-narration of their actions during the test. 

Examples of research methods that are often unmoderated include user surveys, first-click tests, A/B tests, and user analytics.

In moderated research, a facilitator observes—either in person or remotely—as participants take part in the study. This real-time moderation allows researchers to adapt their script and process in response to participant actions and engagement, and also to ask follow-up questions that probe more deeply into why participants make certain choices. 

Because of the human element of introducing a facilitator, moderated tests can be a more time consuming and more expensive form of user testing. They can also require additional expertise and preparation since there is a specific skill set required to moderate a study effectively. And because the output usually includes a good deal of qualitative data, analyzing the results can also take a little more time.

Examples of research methods that are usually moderated include interviews, ethnographic studies, focus groups, and task analysis.

Which user research method should you use?

The answer to this question is, of course, “it depends.”

The right user research method for any given study depends on a combination of factors, including the stages of product development you’re looking for insights about, the nature of your research question, the types (and number) of participants you’re looking to recruit, and your budget.

There are several good frameworks out there to help you choose the right user research techniques and methods, and we cover them in-depth in the upcoming chapter on how to choose a user research method.

💡Not sure which UX research method to choose? We created a free tool that can help.