The concept of ‘enough’ is something we struggle to learn as children—that’s enough ice cream, that’s enough time at the park, that’s enough TV for the day—and sometimes struggle to grasp well into our adulthoods, too. (I, for one, am still learning how much ice cream is ‘enough’ ice cream. Ben and Jerry’s says a half cup. No, that can’t be right.)
‘Enough’ is an important concept in UX research as well. Too much research, too many researchers, too much time and budget, and too many participants can all have a detrimental effect on your results—and too little of all of the above can be even worse. Finding your ‘sweet spot’ will help you discover the best insights with the most efficiency.
So when it comes to recruiting usability study participants, how many is ‘enough’?
Let’s dive into the industry recommendations for the right number of participants, and then look at the factors that might drive that number up or down.
Rather listen that read? Check out this episode of our Awkward Silences podcast, which takes a deep dive into sample sizes.
The short answer: It depends on what you’re trying to learn.
“It depends” is my least favorite answer to any question. But in this case, it really does depend.
Before running a usability study, you’ll map out the specific goals and questions you’d like to answer in your user research plan. These goals will dictate whether you conduct a qualitative or a quantitative study, which will impact the number of participants you need to recruit.
Quantitative studies require more participants to achieve statistical significance. There are many different formulas and approaches to calculating the correct sample size for a quantitative study, but that’s a topic for another time. (If you came here looking for recommendations for a quantitative study, Nielsen Norman Group says 40 participants is usually appropriate, with some exceptions.)
In this article, I’ll focus on how to choose the right sample size for a qualitative usability study.
TL;DR—you’re probably set with 5–10 participants. Here’s why.
Determining your sample size: 4 key considerations
The right number of participants for usability studies is still a widely debated issue. Some researchers insist that the ‘right’ number of participants exists on a scale, determined only on a case-by-case basis. For example, Janet M. Six and Rich Macefield in UX Matters say:
“Generally, the number of participants should increase with study complexity and product criticality, but decrease with design novelty.”
Let’s break down the different factors in this scale and how they impact the ideal number of participants.
1. What is the potential impact of the usability test?
- Higher impact = more participants.
- Lower impact = fewer participants.
The first thing you’ll want to consider is impact.
Some studies are naturally higher-stakes than others—for example, if you’re testing a core product feature that would affect many users, then this test will have the potential for a greater impact than if you were testing something relatively small, like a button in the footer of a webpage.
For higher-impact tests, you might choose to recruit more participants to cover your bases and ensure the most accurate data. Lower-impact tests provide you with a little more wiggle room, so you can justify using a smaller number of participants for the sake of moving quickly.
2. How many rounds of testing do you intend to do?
- More rounds = fewer participants.
- Fewer rounds = more participants.
Usability studies aren’t always one-and-done endeavors. If you have the time and the budget to run multiple rounds of testing, that’s often a better option than running one large test.
This iterative approach will give you opportunities to discover additional insights during later rounds of testing, so you can start out with fewer participants. Start with one small pool of participants for the first round, then conduct additional rounds of testing until you reach statistical significance.
3. How complicated is the task?
- More complicated = more participants.
- Less complicated = fewer participants.
If the task in your study is more complicated, then that means there are more opportunities for errors, inconsistencies, and other mishaps that might skew your results. For example, a usability test that asks participants to go through the full process of completing a purchase—from the homepage to the payment—will have more opportunities for error than a test determining whether or not the ‘sign up’ button is clearly visible on a specific page.
Therefore, the more complicated the task, the more participants you’ll want to recruit. This will give you more accurate data and more flexibility to identify and eliminate outliers.
4. What stage in the product development cycle?
- Earlier stage = fewer participants.
- Later stage = more participants.
The earlier you are in the development cycle, the more likely you are to discover severe, show-stopping errors. Because of their severity, these errors are usually discovered fairly quickly, so you can get away with recruiting fewer participants for early-stage usability studies.
As you get deeper into the development cycle, your product becomes more fine-tuned and the errors less obvious. In this case, you might want to use more participants to help discover any small, lingering issues before you launch.
Okay, but what are the industry benchmarks?
Given all of these considerations, many leading UX researchers have set precedents for the right number of participants—although these precedents may vary slightly depending on who you’re talking to.
Nielsen Norman Group says 5 is enough (except when it’s not):
“The main argument for small tests is simply return on investment: testing costs increase with each additional study participant, yet the number of findings quickly reaches the point of diminishing returns. There's little additional benefit to running more than 5 people through the same study; ROI drops like a stone with a bigger N. And if you have a big budget? Yay! Spend it on additional studies, not more users in each study.”
However, David Di Sippio, Design Researcher and Psychologist, says 3 participants might be plenty for yielding great outcomes using his lean usability testing approach:
“Five people can help you find 85% of the usability problems. However, speaking to three people will generally help you find more problems than you can fix.”
This “lean” approach to usability testing is a highly effective way to improve the user experience with few resources, but it’s also dependent upon the researcher’s expertise and ability to turn insights into action. As UX Researcher Victor Yocco says:
“Researchers must also know how to make the most of the data they collect. Your sample size won’t matter if you haven’t asked good questions and done thorough analysis.”
Moral of the story? You’re probably safe with 5–10 participants.
For most qualitative usability research, 5 to 10 participants is a good baseline—and the considerations listed above will help you determine where your ‘sweet spot’ falls within that range (if not slightly outside of it).
Now that you’ve nailed down your sample size, there are still other things to consider before you get started. Check out our Qualitative Usability Testing Launch Kit for tips, templates, and other resources to get your study off the ground.
If you’re ready to start recruiting, we can help. With a pool of more than 850,000 participants, User Interviews can help you recruit and manage nearly any target audience for your next study. Sign up today and get your first three participants for free.
_1.png)
