How to Choose a User Research Method

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Note to the reader:

This part of the field guide comes from our 2019 version of the UX Research Field Guide. Updated content for this chapter is coming soon!

Want to know when it's released?
Subscribe to our newsletter!

Getting started with user research can be intimidating. Even for experienced UX researchers, the variety of possible approaches to any given research project can feel daunting—with so many methods, which one should you choose? What is the best way to answer your research question? Is it the most efficient way that will keep your project within scope, under budget, and on time? 


In the previous two chapters, we gave an overview of the different types of user research methods, and dove deeper into the important differences between qualitative and quantitative research. You may already have an idea, based on what you read and the pre-existing UXR knowledge you’re bringing to the table, which methods you want to use for your current project.


In this chapter, we’re going to help you solidify those choices, and introduce you to a few different frameworks for choosing the right user research method for any project that comes your way, or any research question that might crop up.


These frameworks—which can be adapted to your own research practice and the realities of your organization, customers, and unique constraints—will help you answer the question “which UX research method should I use?” efficiently and effectively every time.

In this chapter:

  • Start with clearly stated research questions and goals
  • Pick the right method for your research question
  • User research frameworks
  • The right methods for each stage of product development 
  • Mapping UX research methods to decisions
  • A 3-dimensional framework for UXR 
  • Additional considerations


Without further ado, here’s how to choose the right methods for your UX research study.


Start with clearly stated research questions and goals

As ever, you’ll want to start with well-defined goals and clear research questions. 


If you’re not already clear about the goals of your research, stop here. Do not pass go. Do not collect $200 in gift card incentives.


Circle back to the chapter on creating a user research plan for help identifying your research goals, which should be statements about what you’re trying to learn from your research.


To define your research goals, ask yourself:

  • What do I want to know?
  • What don’t I know?
  • How will I know when I’ve learned it?
  • What company goals will this work support?
  • Where am I in the product development process?
  • What decision will this research enable? 
  • What are the anticipated outcomes of this research?


You’ll want to further refine the answers to these questions through stakeholder research—your key stakeholders can clarify business goals and the kinds of decisions your research needs to enable to be successful. 


Good research questions are specific, actionable, and practical. They also contain clues about who you need to recruit and which methods you’ll need to use. 


Some example of good research questions:

  • Are our customers able to successfully navigate to the support page on our site?
  • What are the primary motivating factors behind the decision to purchase pet insurance?
  • What tools do college students use to keep track of their schedules?

‍

đź“– Read more about Planning UX Research

‍

Pick the right method for your research question

In the likely event that you aren’t binge reading the UX Research Field Guide in a single sitting, it’s probably helpful to take a moment to revisit the different types of user research, along with the different questions they can help you answer. 


User research methods can be divided along several different axes. They can be:

‍

Quantitative or qualitative

Qualitative UX research involves collecting data through direct observation of a small group of people in order to assess behavior and provide context. Qualitative methods produce unstructured or semi-structured observational findings like comments, preferences, and motivations.


Qualitative research helps you answer the questions: 

  • “Why?” 
  • “How?”
  • “How can we fix this?”


Quantitative UX research, on the other hand, involves collecting data from a much larger group of people in order to quantify a problem and uncover patterns through statistical analysis. 


Quantitative research helps you answer the questions: 

  • “How much?”
  • “How many?” 
  • “How often?”

‍

Generative or evaluative

Generative (discovery) UX research (also foundational, or exploratory research) uses direct observation, deep inquiry, and careful analysis to generate ideas (go figure) and discover opportunities for innovation that will meet a specific and real need in the market. Generative research is often (but not always) qualitative. 


Discovery methods help you answer questions like:

  • Who are our users?
  • What are their problems?
  • How do they think?
  • Is there a real need for this solution?
  • What should we build?


Evaluative UX research is used to evaluate people’s responses to a product or solution. It is used throughout the product development cycle to test and validate the appeal, intuitiveness, and functionality of ideas, prototypes, and finished products.


Evaluative methods help you answer questions like:

  • Which design do users prefer?
  • Is this interface intuitive?
  • Does this feature work as intended?
  • Are we building the right thing, correctly?
  • What could be better?


Attitudinal or behavioral

​​Attitudinal UX research methods (like surveys and focus groups, for example) rely on self-reported data about people’s beliefs, perceptions, and expectations. Take what participants say with a grain of salt, especially if they’re referring to future behaviors—even the most self-aware user can’t predict the future.


Attitudinal research can help you answer questions like:

  • What do people think about this feature?
  • What do people say they want?
  • How do people describe their current problem?
  • What are their mental models?


Behavioral UX research methods (like A/B tests and first click tests) are based on direct observations about how a study participant interacts with a prototype or finished product. 


Behavioral research can help you answer questions like:

  • How do users interact with this new feature?
  • How long does it take users to complete a workflow?
  • Which CTA has a higher conversion rate?
  • Can people navigate the UI?


User research frameworks

A user research framework is a systematic way of categorizing research methodologies and approaches to guide decisions about which method to use, when. Whether you’re a solo UX researcher or are working within a larger team, a reliable framework will provide consistency and can dramatically speed up research planning and decision-making.


They can be intricate, like this behemoth from UX researcher Lena Borodina:

large ux research methods selection framework
Source


Or they can be simpler, like Tomer Sharon's Lean User Research framework:

‍

Source


The best framework is the one that works for you. In order for it to be used effectively, a framework needs to be relevant to your team, your business goals, and your internal resources. If you don’t have the budget or people power for long ethnographic field studies, don’t include those types of studies in your framework. (Remember, frameworks can evolve.)


All of the frameworks in this chapter can and should be adapted to your own practice as necessary. We’ll cover how to map UX research methods to:

  • Where you are in the product development cycle
  • The decisions you want to enable
  • The nature of the data collected


Let’s hop to it!


The right methods for each stage of product development 

In the very first chapter, What Is User Research?, you learned about how (and why) to conduct user research at different stages of the product development cycle.


As you progress through the discovery, design, testing, implementation, and post-launch phases, the problems you need to solve will change. You’ll ask different questions, which will require different research methods to answer.

ux activities in the product and service design cycle
NN/g's UX Research Cheat Sheet

‍

Discovery stage (pre-prototype)

Most projects begin with some discovery, in which you’re trying to pinpoint the problem and get a clearer picture of who you’re solving it for. 


The goals at this stage are to explore new directions, develop a detailed understanding of (potential) user needs and their context, and uncover opportunities. Both qualitative and quantitative approaches are used during this phase.


Conducting UX research during discovery helps you empathize and strategize.

‍

Key discovery methods

Stakeholder Interviews—Understanding what your stakeholders ultimately value and need to know is a necessary component of effective user research. Part of a user researcher’s role is to align and articulate both business and user goals, and find where they intersect. 


Ethnography—Ethnography is all about observing people and their habits in context (their natural habitats, if you want to get zoological about it). Field studies and online ethnographic studies can help you understand how people behave in groups and how they might interact with your product outside of a lab environment. What people do is so often different from what they report, and ethnographic studies can be a great way of getting around this issue.


Diary studies—When you need to understand long-term user behavioral patterns, diary studies are a great option. They provide insights into habits, changes over time, motivations, and long-term customer journeys. And they can be a solid alternative to ethnographic field studies when you’re on a limited budget.  


Focus groups—Focus groups are not exactly the, ahem, darling of UX research circles these days. But they have their uses, especially when it comes to market research. Focus groups allow you to observe a lot of people relatively quickly. When you are very early in developing and marketing your product, this type of research can be useful in getting a broad view into the audience you seek to serve. 


Generative user interviews—In-depth user interviews typically involve talking to participants one-on-one, asking them a set of non-leading questions, with an emphasis on past behaviors and perceptions. User interviews are valuable throughout the product development cycle, but are especially important in the early stages—the things you learn might confirm your hypotheses or lead to wildly different conclusions that reshape the direction of your inquiry.

‍

đź“– Read more about Discovery Research Methods

‍

Concept validation and testing stage (prototype and build)

At this point, you have a good sense of your market, your users, and their needs. You’ve come up with an idea for a solution, are actively building (or have built) a prototype, and are ready for feedback.  

‍

This stage of product development is about evaluating your solutions and validating your decisions through both qualitative (formative) and quantitative means. The goals of testing are to validate conceptual fit, evaluate the usability of the design, and inform changes.

‍

Eventually, you will reach a "go/no-go" decision point, when you transition into a period in which you are continually improving the design direction that you have chosen to reduce the risk of execution.

‍

Conducting UX research during this phase helps you optimize and execute.


Key validation and testing methods

Qualitative usability testing—This method involves having participants think aloud as they interact with a prototype or product, allowing researchers to evaluate implicit and explicit cues and find patterns quickly. 


Task analysis—Task analysis helps you better understand your user’s goals, and how they go about achieving them. This can be coupled with a variety of other methods, and is an important aspect of evaluating the overall user experience for most apps.


A/B testing and multivariate testing—Whether you use A/B test, A/B/C test, single variant test, or multivariate test, the goal is to collect quantitative data on which version of your product best achieves the goal of the test—generally a hybrid of a business and user goal, for instance conversion rate on an ecommerce site.


First click testing—This is just what it sounds like. First click testing—in which you record the first click someone makes to accomplish their goal and analyze the results—is a popular way to assess the user’s ability to efficiently and effectively complete a task.


Card sorting and tree testing—Card sorting and tree testing are methods for testing your information architecture (IA), i.e. how you categorize and label content. Card sorts, which involve users grouping topics into categories, can provide great insight into your user’s mental models and how you should organize your content. Tree tests—in which users are tasked with finding information in a sitemap—are often used as a followup method to evaluate how intuitive your site navigation is.


Accessibility testing—The validation and testing stage should not be the first time accessibility crosses your mind; —accessible design begins with thoughtful planning and research design.  By the time you get to this stage of product development, you should be using accessibility testing to stress test prototypes or products that have already been designed with accessibility in mind. In addition to running your product through accessibility checking tools, consider conducting qualitative usability studies with participants with disabilities.

‍

đź“– Read more about Evaluative Research Methods

‍

On-going listening (post-launch) 

You’ve launched a product… congratulations! 

‍

Now, keep researching it!

‍

No, seriously—user research doesn’t (or shouldn’t) stop once you’ve launched a product or feature. “Set it and forget it” is not a smart product strategy. Ongoing listening methods help keep your product useful, impactful, and relevant over the long-haul.

‍

The goals of post-launch research are to understand how well your product is performing, assess how well it continues to address customer needs, and identify opportunities for improvement. Methods are typically quantitative (summative) in nature, and allow you to measure product performance against itself or its competition.

‍

Conducting UX research after launch helps you assess and iterate.

‍

Key ongoing listening methods

Feedback surveys—There’s a variety of survey types and tools that can help you meet your goals, from user intercept (in-app) surveys, to NPS surveys, to longer surveys that dig into specific aspects of the user experience. Like other methods at this stage, ongoing listening surveys typically require some cross-functional collaboration to do effectively. Talk to your product team to find out where and when an in-app survey would be most effective, partner with marketing to launch and monitor an NPS survey, and coordinate with customer development to segment and send email surveys to existing customers.


Analytics—Analytics can be a treasure trove of great quantitative data if you know how to analyze them and connect insight to action. Keeping a close track on key user flows, in-app behaviors, and business metrics will help you see changes over time. 


Bug reports and support tickets—Bugs come up and need to be fixed. That’s just the reality. Make sure you have good bug reporting and systems to address them rapidly. Your dev team is probably on top of this, but you can use this data to understand where current or historical frustrations may have impacted the user experience in ways you wouldn’t have known otherwise. Similarly, FAQ and support desk reporting can help you understand where users are getting stuck. Taking proactive steps to clear up these moments in the product itself will improve the user experience and make your support team oh so happy.


đź“– Read more about Continuous Research Methods

‍

Mapping UX research methods to decisions

Research is systematic inquiry. If it is not systematic, you’re not researching—you’re just being a busybody. To quote Roberta Dombrowski, VP of User Research at User Interviews:

‍

"Research is the act of acquiring knowledge—we could literally conduct research on any topic under the sun. That endless potential scope is why it's so important that, as practitioners, the research we are leading and conducting is focused and aimed at enabling decisions for our organizations."

‍

At User Interviews, we keep our research focused by using our own Decision Driven Research Framework, which is centered on the belief that the purpose of research is to enable decisions for your company, product, or service. 

‍

A framework for Decision Driven Research

In this framework, the idea is to map your research methods to the types of decisions you want to enable. There are 4 types of decisions:

  • Vision decisions establish a potential company, product, or service direction. To enable vision decisions, you should choose methods that give you clarity on participants’ big-picture beliefs, philosophies, and experiences.
  • Strategy decisions determine how you will achieve your vision. To enable strategic decisions, you should choose methods that give you detailed insights into participants’ big-picture beliefs, philosophies, and experiences.
  • Definition decisions determine whether or not you pursue a specific design direction. To enable these decisions, you should choose methods that allow you to get early feedback on potential design directions and to better understand how a participant may interact with a potential product or service.
  • Evaluation decisions concern iterations to existing products or services. To enable evaluation decisions, you should choose methods that allow you to continuously identify problems, bugs, or confusion that customers may encounter during use.


Below is a list of the most common methods we use when conducting research at User Interviews. For each method, we’ve created internal documentation about what the method is, the type of decisions it helps inform, and the output.

a table mapping research methods to decision types

‍

There are 5 stages to doing Decision Driven Research, the first three of which are all about creating a clear and focused plan for research. Here’s the process in full:


  1. Identify the decision trigger. This is the first and most important phase, in which you identify that there's a decision that your team has to make, or a question you need to answer. 
  2. Scope evidence. Once you know what decision needs to be made (i.e. once you’ve defined your research goals), take time to review whether there is any existing data or evidence that can inform the decision or even answer your research questions.
  3. Define your approach. If there's no existing data or you decide you need more information in order to proceed, then it’s time to figure out which approach or research method will get you the answers you need to confidently inform a decision. 
  4. Explore. Now that you have a research plan, it’s time to go out, explore, and gather data. In other words, research!
  5. Reflect and decide. Finally, analyze and synthesize, reflect on what you learned, and make informed decisions about what to do next. 

‍

đź’ˇ Read more about the Decision Driven Research Framework

‍

A 3-dimensional framework for UXR 

Many teams limit their research opportunities by consistently using the same familiar methods over and over again. And while it’s unlikely you’d ever need to use all the different UX research methods out there on a single project (what a nightmare), you should be prepared to mix up your playbook when the research question calls for it.


Nielsen Norman Group developed a helpful chart that illustrates where 20 popular methods fall along the following axes:

‍

Attitudinal vs. behavioral—Attitudinal refers to what people say, while behavioral refers to what they do. Often there’s a pretty wide gap between the two. The most popular research methods blend elements of these two to capture both what people say and what they do. 


Qualitative vs. quantitative—As we’ve been discussing, qualitative research data comes from observing behaviors and/or attitudes directly while quantitative research data is acquired indirectly via an instrument like a survey or web analytics tool. 


Context of use—The final axis of NN/g’s three-dimensional framework looks at how study participants use the product in question (if indeed they do). Context of use can be defined as:

  • Natural or near-natural use of the product 
  • Scripted use of the product 
  • Not using the product during the study
  • A hybrid of the above


Using this chart, you can start to hone in on which methods will satisfy your research goals: 

‍

landscape of user research methods chart from nng
Source


The folks at NN/g are careful to note that theirs, like all frameworks, is not set in stone. Few of the methods are fixed in a single place.


Most of the methods in the chart can move along one or more dimensions, and some do so even in the same study, usually to satisfy multiple goals. For example, field studies can focus on what people say (ethnographic interviews) or what they do (extended observations); desirability studies and card sorting have both qualitative and quantitative versions; and eye tracking can be scripted or unscripted.


Additional considerations

User research rarely (if, indeed, ever) happens without constraints. You may be short on time, money, or people power. And that will impact which methods are available to you.


Some methods are expensive and time consuming

Say you want to learn about how participants interact with your product in a real-world context. Field studies would be an obvious choice—you could observe people in the field, using your product IRL. Great! 


But field studies can take months. 

  

If you don’t have the luxury of time, diary studies would be a solid option for learning about product use in-context. The best UX research method in this case is the one you can reasonably fit into your timeline and budget.

‍

Some methods require more people power

Unmoderated research involves unobserved tests that a participant can complete at their own pace using tools that prompt them to answer specific questions or perform specific tasks. The nature of this kind of test makes it easier, less expensive, and faster to run than research facilitated by a professional moderator. 


Moderated research requires a moderator (go figure). Researchers observe as participants complete a study, which allows researchers to respond to participant actions on the fly and to ask follow-up questions that probe more deeply into why participants make certain choices. 


Because of the human element of introducing a facilitator, moderated tests can be more time consuming and more expensive. They can also require additional expertise and preparation since there is a specific skill set required to moderate a study effectively. And because the output usually includes a good deal of qualitative data, analyzing the results can also take a little more time.


You shouldn't skip moderated research (which includes methods like interviews and qualitative usability testing). You really only need about 5 participants, anyway. But if you’re trying to collect data at scale, whether that’s qualitative or quantitative, remember that there’s only so much time in a day. Be realistic about how many facilitated research sessions your team can handle within a project’s timeline.

‍

Some methods need specialized tools

We don’t have a usability testing lab at User Interviews. Nor do we have a closet full of eye tracking devices or other biometric technology. In fact, we don’t have a lot of the things some user research teams rely on as part of their practice. 


And that works for us! We’re a fully remote company, and we conduct remote UX research. We don’t need to monitor people’s heart rates or do research in clinically controlled environments to answer our research questions.


We also don’t pay for all-in-one research platforms. That’s less about cost, and more about the fact that we’d rather assemble our own toolkit with products that let us bring our own participants (recruited with User Interviews, naturally). 


But that’s just us. The tools you need for user research might be different. Or not—many user research methods don’t require special software. 


If your research question can be answered by simple means—do that. Research that’s hard to do doesn’t get done.

‍

So, which user research method should I choose?

Well, that was kind of the whole point of this chapter. But if you’re still feeling stuck, we created a methods quiz that might help! 

‍

âť“Take the UX Research Method Quiz

‍

Editor’s note, December 2021: We’re in the process of updating the resources associated with each method recommendation in the UX Research Method Quiz. 


Keep reading to learn more about each user research method in depth. First up: Discovery research methods!

Recruit from our panel of 700,000+ vetted participants

Sign up for free

Subscribe
X

Master UX Research

Get the full course and exclusive pro tips delivered straight to your inbox. (New content dropping in 2022)!
Thanks! See you in your inbox 👋
Oops! Something went wrong while submitting the form.