All posts

UX Research Topics

Podcasts

Field Guide

SUBSCRIBE TO OUR NEWSLETTER

Thank you! You are all signed up.
Oops! Something went wrong while submitting the form.

FOLLOW US

BlogRecruiting

July 2, 2020

  • Last Updated:
  • July 2, 2020

Screening Participants in Research: How to Design Your Questions, Pitch Your Target Audience, and Choose the Right People

Decide on the right screening criteria, write non-leading questions, and identify the right mix of great research participants.

Olivia Seitz

Screening participants in research is how you determine whether a person is a good fit for your study. As any good researcher knows, it sounds simple enough ... but putting into place a robust screening process that consistently results in the right participants is actually complex. In addition, the steps you can take to populate your study depend on whether you’re doing medical research or user research.

For user researchers, screening can proceed seamlessly with recruitment efforts. Our guide to participant screening was written primarily for user experience researchers since that’s our area of expertise, but many of the principles (from how to write your questions to ensuring candidate diversity) apply for both types of research studies.

Here’s a quick look at what we’ll cover:

  • How to decide on your inclusion and exclusion criteria
  • How to screen for demographics, geographics, behaviors, and psychographics
  • Some thoughts on how to get a diverse panel
  • How to write non-leading questions that weed out “professional testers”
  • When double-screening candidates with a phone call makes sense
  • Extra requirements for medical research screening
  • The easiest way we know of to find high-quality research participants.


Note: Looking for a specific audience to participate in your user research? User Interviews offers a complete platform for finding and managing participants in the U.S. and Canada. Find your first three participants for free. Or, streamline research with your own users in Research Hub (forever free for up to 100 participants).

The best stories about user research

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

What Do You Need to Know?

Before you write your first screening question, think through who belongs in your study and who doesn’t. Here are a few activities we recommend to help you consider your criteria:

  • Write down what you’re trying to determine (and if that goal isn’t clear, start by refining the scope and goal of your research). Articulate the research question you want to answer
  • Talk to your study team (if you have team members), and interview key stakeholders about what they hope to learn from the research
  • Based on your answers from the above exercises, write out all the traits and behaviors of your ideal target audience (example provided below)
  • Write down the traits of people who would not use your product or who you already know are not a good fit for it

When you’re finished, you should have a clearer picture of what you’re screening for and against.

For example, let’s say you’re developing an app to help disabled diabetes patients manage their health by teaching them exercises they can do while sitting.

So you might rule out anyone who is able to walk on their own. You might decide to include potential subjects who have had an amputation in the last six months of their medical history. You would, of course, only want to include diabetes patients.

Here's an example of writing inclusion and exclusion criteria for screening participants in research.


But there are other traits where the answers aren’t as clear. Are you looking for people who have tried and failed to exercise consistently in the past? Do you want recently diagnosed patients, or can study participants have obtained their diagnosis years ago? How motivated to get healthier should they be?

As you dig into these questions, you might find it helpful to segment your criteria according to four categories: demographics, geographics, psychographics, and behaviors. 

Demographics, Geographics, Psychographics, and Behaviors

Demographics include objective facts about the prospective subject, such as ethnicity, gender, age, education level, marital status, occupation, and income. Geographics cover where your users live and work. Psychographics include their activities, interests, and opinions. Behaviors include how and why they would interact with your product or service.

The first two are easy to screen for. Continuing the previous example, if your training app is intended for adults in the U.S. and Canada who speak English or Spanish, then you’ll want to look for traits like:

  • ≥18 years old 
  • Lives in the U.S. or Canada
  • Speaks English or Spanish.

If income and occupation don’t matter, don’t create screening questions for them. Leaving other demographics questions open will help you select a diverse set of research subjects later in the process.

Screening for psychographics and behavior is more nuanced. We’ll get into the details of how to write those questions later in the post, but here’s an example of each:

Behavioral screening question example: 

How often in the past six months have you broken a sweat while working out?

  1. Not at all
  2. At least once per month
  3. At least once per week
  4. 5-7 days per week  

Psychographic screening question example:

When you think about spending 30 minutes per day working out, how does that make you feel? (Free Response)

The behavior question attempts to determine their history of exercise without giving away the “right” answer. If you’re looking for people who haven’t exercised much, then A) and B) might be your preferred answers. But if you want someone who’s made a consistent effort to exercise, then C) or D) would be appropriate.

The psychographic question digs into how they feel about exercise: Does it seem daunting to them? Does it seem doable? If you want people with a certain view regarding exercise, this is the way to find them.

Quotas: Ensuring Your Study Is Diverse

Screening Participants in Research


One of the dangers to avoid when designing your screening process is overscreening. It’s easy to accidentally screen out prospective participants by making your criteria too precise. By staying open to people with different demographic, psychographic, and behavioral backgrounds, you can ensure that your research represents the diversity of your possible users.

One of the best ways to ensure you have a diverse and inclusive pool of participants in your study is to comb through your screener responses and approve participants manually until you have the spread you want.

Having a diverse mix of participants is important for generative research and usability studies alike. Otherwise, the product your team makes may not be usable by certain segments of the population. That’s not to say you should include demographics that are clearly not a good fit for your work (e.g. talking to low-income participants when you’re developing luxury goods), but it is worth taking the time to ensure that a mix of ethnicities, educational backgrounds, and so forth are represented in your study, especially groups who have been historically underrepresented but may be great users and customers for your product!

To learn more about historical, systemic failures to include women and people of color in medical research and product development, consider listening to this episode of 99% Invisible. The consequences of a failure to include diverse participants can be concrete — money, time, and even lives can be at stake. For our part, we are researching implicit bias in our product to help foster more diversity and inclusion in research sessions.

Ethnicity Distribution of 2020 Signups: 10.7% Asian, 16.3% Black, 8.9% Hispanic or Latino, 10.9% No info, 53.2% White
Distribution of 60,000 participant signups for User Interviews in 2020.


Something else to consider is testing with folks who have different situational, temporary, or permanent disabilities that affect how they access your product. What if they’re blind? Missing a limb? Constantly distracted? How can you design the experience to be easier and more intuitive for them?

Inclusive design for Permanent, Temporary and Situational circumstances.
An illustration by Henry Neves-Charge from his post on inclusive design.


There are other types of diversity to consider, too. If you’re doing usability testing for a web app that still displays well on phones, for example, then you might talk to users of different browsers and operating systems (iOS, Android, Chrome, Firefox, Edge, etc.).

Finally, one of the best ways to ensure you’re inclusive is to also make sure the screener questions are as free from bias as you can manage; we’ll chat about best practices for writing those questions next.

How to Write Questions for Screening Research Participants

Here are a few of our favorite practices for writing unbiased screening questions to help you find the best participants for your study.

How Many Questions You Need

We recommend writing at least one question per eligibility criteria you’ve identified. That said, we’ve noticed that researchers more often write screener surveys that are too long versus too short.

The longer your screener, the less likely potential participants will make it until the end. And if your screener is too long, you may confuse participants who thought that they were already participating in your research by answering the screening questions!

If you’re struggling with survey length, try to make your criteria less restrictive and eliminate any repetitive questions.

Avoid Leading or Loaded Questions — and Don’t Give Away the ‘Right’ Answer

A leading question contains language that subtly influences the choice of the participant. It might make an assumption about the respondent’s opinion or use non-neutral language. Here’s an example that researcher Els Aerts shared with us from Twitter:


Do you love working remotely? (Yes, No, I never work remotely.)


The question asks, “Do you love working remotely?” The question is phrased in a highly positive way, thereby encouraging highly positive responses.

It’s all too easy to bake your hoped-for response into the manner in which you ask the question. Returning to the exercise app example from earlier in this post, which of these phrases is leading vs non-leading?

Do you need assistance to walk? (Yes, No, Sometimes.)
Which of the following best describes your ability or inability to walk around? (Able to walk with assistance, able to walk without assistance, bedridden, able to move around in a wheelchair.)


The first question, “Do you need assistance to walk?” could imply that you’re looking for participants who are not fully ambulatory. The second question uses neutral language and offers answer choices that don’t reveal the intended response.

If you’re struggling to determine whether or not your questions are leading, ask yourself:

  • Do I use neutral adjectives?
  • Do users have sufficient answer choices to respond honestly?
  • Are the answer choices equally weighted?
  • Am I making any assumptions about their behavior or preferences?
  • Have I included unnecessary modifiers or descriptions that introduce the opportunity for biased responses?
  • Have I tested these questions on colleagues or a small group of users?

Here’s one last example to help. If you want to talk to diabetes patients, it’s important not to state that outright, if you can. So you might ask a question like this:

Select all chronic health conditions (if you have any) that you have been diagnosed with.

  • Heart disease
  • Diabetes
  • Hypertension (high blood pressure)
  • Kidney Disease
  • Arthritis

This kind of questioning helps you identify folks who have diabetes without leading respondents to say they have it.

Use Accessible Language

Screening surveys are not the place to use flowery, fancy language. Avoid less widely used words—for example, “ambulate.” Also, avoid idioms and slang (such as “beat around the bush” or “cut corners”); these phrases are often difficult for non-native speakers to understand.

Screen for Articulation

How do you feel about engaging in moderate exercise for 30 min or more per day? Be as specific as possible.


We recommend including at least one open-ended question that forces respondents to articulate their opinion or behavior. It’s a good way to see who can give detailed and insightful responses versus folks who maybe won’t be the most helpful when you’re asking them questions in the study itself.

It’s also a good way to know if someone is who they say they are; if they’re able to speak about a topic knowledgeably, they’re likely not just answering whatever they think you want to gain access to the study incentive.

Here are some example articulation questions:

  • How do you feel about engaging in moderate exercise for 30 min or more per day? Be as specific as possible.
  • Think of a task at work that you dislike. Why do you dislike it? Please be as specific as you can.
  • How do you decide what to buy at the grocery store?

How to Pitch Your Study to Potential Participants

Even when you’re finished crafting your screening survey, the way you talk about your research to potential participants matters.

Revealing the name of your company (if you’re not communicating directly with existing users) or the exact purpose of the study can bias participant responses. In the case of the exercise app for diabetes, you might tell participants it’s a study related to exercise habits or a lack thereof. That way they have some context (which can help them decide to click into the screener), but they don’t know what type of exercise habits you’re looking for.

This will make it harder for professional testers to guess what you want, making it more likely you’ll get authentic responses.

When Double-Screening Makes Sense

If you want, your screening procedures can include a phone call to the potential candidates who seem most promising. Double-screening like this often isn’t necessary, but it’s a good way to be absolutely sure that you’re getting the right people to talk to during your study.

We’ve seen researchers use double-screening when the study they’re doing is high-profile (visible to important stakeholders in their organizations) or when they have a highly specific research need. Sometimes, they make the calls just to know that they won’t have inarticulate or untrustworthy participants.

If you use User Interviews for double-screening, you will be given the participants’ phone numbers or email addresses after they’ve completed the screener.

Extra Requirements Faced by Medical Researchers

I Agree


If you’re conducting medical research, the recruitment process and screening process are considered separate activities. Recruitment — in which you reach out to research candidates and tell them about the planned study — is a pre-screening activity that can be done without informed consent. But even your pre-screening process may have to be submitted to your Institutional Review Board (IRB) before you can proceed.

Before you gather protected health information or obtain medical records to determine study eligibility, you’ll need patients to sign a consent form to proceed with screening activities. Your screening script for interacting with possible participants and gathering information also has to be submitted for IRB review.

For more information about IRBs, refer to the FDA website. For your institution’s specific rules regarding screening and research procedures, refer to its specific IRB.

Recruiting UX Research Participants with User Interviews

Getting your screener in front of the right people can take weeks (along with some clever advertising and outreach) unless you use a service or platform to do the work for you. At User Interviews, we offer fast research recruiting combined with high-quality participants. Our median time to match researchers with their first participant is just two hours. 

We offer two recruiting solutions: Recruit (which pairs you with participants who aren’t necessarily existing users) and Research Hub (which helps you manage research with existing users of your product).

Recruit includes our panel of over 350,000 vetted professionals and consumers; if your niche is too narrow for our existing base, we’ll recruit more. You can quickly find participants from 500 different professions, in locations throughout the U.S. and Canada, of any ethnicity and income level, and more. Use our screening tool to look for specific psychographics and behaviors.

But if you already have users and just need a better way to manage doing research with them (without stepping on your team’s toes), consider Research Hub. We provide automated tracking of research activities (like who’s been contacted, whether they participated, and what they’ve been paid in incentives). Reminders for participants, incentive payouts, and scheduling are all automated, just like in our Recruit tool.

User Interviews Research Hub free forever


Best of all? Research Hub is free forever for the first 100 participants you add to the platform.

To get started on Recruit or Research Hub, create a free account (and find your first three participants for free). Launching a study is completely free; you only pay for completed research sessions with the participants we find.

Olivia Seitz

Olivia is a content strategist at Grow & Convert who loves science, cats, and swing dancing. She enjoys a mix of writing, editing, and strategy in every work week.

More from this author