Best Practices

Seven Common Screener Survey Mistakes (And How To Fix Them)

It's easy to create a screener survey, but taking a little extra time can drastically increase your recruit's effectiveness.

Melanie Albert
/
March 1, 2018

Having managed thousands of research projects (1,289 to be exact), I’ve seen the full spectrum of recruits. From dazzlingly easy recruits to those that take a bit longer to fill. Through them all, I’ve gained enough experiential knowledge to know what works, and what doesn’t.

Since the operations team is responsible for client success, part of our job is to inform clients of what we’ve learned. Here are seven common screener mistakes and proven suggestions for setting your recruit up for success.

Mistake #1 - Giving it all away too soon

One of the first things you’ll want to avoid is giving away too much information up front. In a nutshell, it devalues the screener process itself. The screener is designed to cleverly find the right candidates who would be a perfect fit for your study.

Let’s say your research is about country music. Rather than this:

… try something like this:

Follow the same advice for your screener survey with your study title, too. Rather than calling this study “Country Music Lovers Only,” you might want to try something more simple and unspecific, like “Music Study”. You’ll be able to pull in a wide audience and carefully sift out the true country music fans. In addition, you’ll gain more information about your participant’s interests if you find yourself wanting to dig into data further.

Mistake #2 - Making your criteria too niche

Looking for oceanographers between the ages of 21-25 who live in Indiana? Yeah, that’s a tricky one. While it’s good to have a distinct audience in mind for your research, considering the feasibility of your recruit is always important. I encourage researchers to have some flexibility around some of their criteria in case the audience is too niche to reach your goals.

Mistake #3 - Making your criteria too broad

On the flip side, you’ll want to make sure that your recruit has focus. Otherwise, you might lose track of what you’re trying to accomplish. Running a study about mobile apps? You could include questions that filter out people who don’t have the smartphone you need—or perhaps don’t have a smartphone at all. Specifying the audience that best fits your research needs will give you a more refined candidate pool that you won’t have to comb through later.

Mistake #4 - Spelling and Grammar Mistakes

Every day we have a mix of projects launched by both experienced researchers, and some who might be recruiting for the very first time. Either way, our team always reviews projects to make recommendations.

Of the missteps that we come across, spelling and grammatical errors are at the top of the list. We know—you want to launch your studies and get the recruit started as quickly as possible, but keep in mind that your screener is the first touchpoint you have with potential participants. Not only are spelling errors jarringly unprofessional, but they can lead to confusion if they interrupt the clarity of the question being asked. Taking a few minutes to review your screener could save you time fixing bigger mistakes later. Try asking a colleague to read your survey, or write it, then come back to it an hour, or even a day later if you have the luxury.

Mistake #5 - Yes or No… or Maybe?

The screener is your chance to get to know the people who might become a part of your user research endeavor. You might as well learn as much as you can about these folks!

One of my personal pet peeves is coming across screeners with exclusively “Yes/No” questions. At User Interviews we offer a variety of question formats, and you can use them all to your advantage. Sure, you could list a bunch of questions with simple quick-click options, but why not add a little fun and nuance into the mix? Plus, adding some more interesting questions can give you more insights than you imagined.

In one study, a client was interested in speaking to people who proactively look for deals when shopping online for clothing. The original screener looked something like this:

This was such a missed opportunity to gain some valuable consumer insights! And the screener doesn’t need to be long to be insightful. Our revisions, pictured below, would add some much-needed clarity to candidate responses.

The outcome of a few minutes of thoughtful revisions? Improved data collection and a happy client!

Mistake #6 - Avoiding Logic (Skip Logic, That Is)

Skip logic is a great way to customize your screener. It offers areas to give some participants an opportunity to expand their responses, as needed.

For instance, let’s say you are conducting a study on pet ownership. You might want to capture more information about the type of pets someone has, but you also don’t want to exclude those who have no pets. In the example below, you’ll notice that the questions on page two are only relevant for respondents who do not have pets. Skip Logic allows you to set up the respondent journey so that pet owners will essentially jump over these questions. It creates a customized feel and avoids confusion for the respondent while gathering the information you need.

Mistake #7  - Not getting to know each other

While the questions on your screener help to strain your candidate pool, many researchers miss out on opportunities to get a sense of a candidate’s personality, and even have a little fun!

I always encourage researchers to include at least one articulation question, typically at the end of the screener. After all, this is a bit of an interview process to see who would be the best fit for your study. Perhaps you’re looking for creative-thinkers to join in an interactive in-person focus group. Use an open-ended question to capture more about the way they approach situations, and think/problem solve. Here are a few of my favorites:

  • You just won $10 million in the lottery. What’s the first thing you’d do, and why?
  • What was the last TV show you enjoyed? Why should I watch it? 
  • You’re a new addition to the crayon box. What color would you be and why?
  • Tell me about a product you recently purchased and liked. Why should I buy it?

Well, there you have it. The seven most common screener mistakes we see in our day-to-day. If you have questions about screener surveys or getting started with your first project at User Interviews, drop us a line at projects@userinterviews.com. Happy recruiting!

Want to contribute to User Interviews content? Here’s how.

P.S. Can we email you?

Get "Fresh Views," our weekly newsletter, delivered to your inbox. We feature interviews and point-of-views from UXers, product managers, and research nerds of all stripes.

Subscribe
Melanie Albert

Melanie Albert is the VP of Operations at User Interviews. She's managed thousands of research studies and knows a thing or two about how to find great participants for user research.

Recent Posts

Subscribe to FRESH VIEWS newsletter

Weekly interviews and point of views all about UX research

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.