Join over 150k subscribers to get the latest articles, podcast episodes, and data-packed reports—in your inbox, every week.
SUBSCRIBE TO OUR NEWSLETTER
Enough’s enough. Here’s how to find your ideal sample size—the “sweet spot” that will deliver quality insights with the most efficiency.
The right number of participants for usability studies is still a widely debated issue. Some researchers insist that the ‘right’ number of participants exists on a scale, determined only on a case-by-case basis. For example, Janet M. Six and Rich Macefield in UX Matters say:
“Generally, the number of participants should increase with study complexity and product criticality, but decrease with design novelty.”
Let’s break down the different factors in this scale and how they impact the ideal number of participants.
The first thing you’ll want to consider is impact.
Some studies are naturally higher-stakes than others—for example, if you’re testing a core product feature that would affect many users, then this test will have the potential for a greater impact than if you were testing something relatively small, like a button in the footer of a webpage.
For higher-impact tests, you might choose to recruit more participants to cover your bases and ensure the most accurate data. Lower-impact tests provide you with a little more wiggle room, so you can justify using a smaller number of participants for the sake of moving quickly.
Usability studies aren’t always one-and-done endeavors. If you have the time and the budget to run multiple rounds of testing, that’s often a better option than running one large test.
This iterative approach will give you opportunities to discover additional insights during later rounds of testing, so you can start out with fewer participants. Start with one small pool of participants for the first round, then conduct additional rounds of testing until you reach statistical significance.
If the task in your study is more complicated, then that means there are more opportunities for errors, inconsistencies, and other mishaps that might skew your results. For example, a usability test that asks participants to go through the full process of completing a purchase—from the homepage to the payment—will have more opportunities for error than a test determining whether or not the ‘sign up’ button is clearly visible on a specific page.
Therefore, the more complicated the task, the more participants you’ll want to recruit. This will give you more accurate data and more flexibility to identify and eliminate outliers.
The earlier you are in the development cycle, the more likely you are to discover severe, show-stopping errors. Because of their severity, these errors are usually discovered fairly quickly, so you can get away with recruiting fewer participants for early-stage usability studies.
As you get deeper into the development cycle, your product becomes more fine-tuned and the errors less obvious. In this case, you might want to use more participants to help discover any small, lingering issues before you launch.
Given all of these considerations, many leading UX researchers have set precedents for the right number of participants—although these precedents may vary slightly depending on who you’re talking to.
Nielsen Norman Group says 5 is enough (except when it’s not):
“The main argument for small tests is simply return on investment: testing costs increase with each additional study participant, yet the number of findings quickly reaches the point of diminishing returns. There's little additional benefit to running more than 5 people through the same study; ROI drops like a stone with a bigger N. And if you have a big budget? Yay! Spend it on additional studies, not more users in each study.”
However, David Di Sippio, Design Researcher and Psychologist, says 3 participants might be plenty for yielding great outcomes using his lean usability testing approach:
“Five people can help you find 85% of the usability problems. However, speaking to three people will generally help you find more problems than you can fix.”
This “lean” approach to usability testing is a highly effective way to improve the user experience with few resources, but it’s also dependent upon the researcher’s expertise and ability to turn insights into action. As UX Researcher Victor Yocco says:
“Researchers must also know how to make the most of the data they collect. Your sample size won’t matter if you haven’t asked good questions and done thorough analysis.”
For most qualitative usability research, 5 to 10 participants is a good baseline—and the considerations listed above will help you determine where your ‘sweet spot’ falls within that range (if not slightly outside of it).
Now that you’ve nailed down your sample size, there are still other things to consider before you get started. Check out our Qualitative Usability Testing Launch Kit for tips, templates, and other resources to get your study off the ground.
If you’re ready to start recruiting, we can help. With a pool of more than 850,000 participants, User Interviews can help you recruit and manage nearly any target audience for your next study. Sign up today and get your first three participants for free.
Content Marketing Manager
Marketer, writer, poet. Lizzy likes hiking, people-watching, thrift shopping, learning and sharing ideas. Her happiest memory is sitting on the shore of Lake Champlain in the summer of 2020, eating a clementine.