Note: Our VP of User Research, Roberta Dombrowski, and Lily Aduana (in her new capacity as our first Research Operations Manager), recently revisited this topic at Rosenfeld DesignOps 2021. You can watch that recording here.
Recruiting participants for research is hard work, and 27% of people who do research use an external recruiting agency to take this time-consuming and tricky task off their hands. But is an agency really the best way to recruit research participants?
At Rosenfeld Advancing Research 2021, we examined five reasons you should consider bringing your recruiting in-house instead of using an agency Our Associate Product Marketer Brittany Rutherford and Senior Project Coordinator Lily Aduana, along with Savannah Hobbs, Design Researcher at Intuit, broke down why recruiting in-house is the way to go.
About our speakers
Brittany Rutherford is an Associate Product Marketing Manager and early employee at User Interviews.Brittany understands the importance and impact research can have on the success of a company—she learned this firsthand at her previous company, a startup that failed because leadership didn’t believe in UXR and built a product no one wanted. She’s enjoyed being on the project coordination side of research and acting as a participant advocate—and she’s always eager to get feedback about UI’s product and user experience.
Savannah Hobbs is a Design Researcher at Intuit working on the Consumer Group team. She also works with the investor & self-employed Credit Karma teams. She manages all aspects of the research—from scoping to recruitment to the final shareout. Savannah relies on recruitment platforms to help her find the best participants to give feedback on concept explorations, or even watch them file their taxes (yes, in real time)! Savannah enjoys collaborating cross-functionally with her partners in product and design to understand the cohort and make actionable recommendations to improve product experience.
Lily Aduana is a Senior Project Coordinator on the User Interviews operations team. Before joining User Interviews, she managed research operations for a UX research agency where participant recruitment and data collection were constantly at play. This experience has given Lily a deep understanding of the importance that participant recruitment has on the flow of research operations and researcher success. Lily enjoys collaborating with User Interviews customers and seeks to add value to UX research by delivering success through creative project coordination, participant recruitment, and research operations.
1. It’s actually quicker to recruit research participants yourself
Only 6% of people who do research plan their research sessions a few days in advance, according to our State of User Research report. Research is something that evolves and changes with the product you’re working on, and it should be able to move quickly. If you’re not able to easily adjust your plans or find participants when you need them, your entire research engine slows down.
So can you recruit research participants quickly? Bringing your research recruitment in-house can seriously speed up your process.
How long does it take to recruit participants through an agency?
First, let’s look at how long it takes to recruit research participants through an agency. Typically, research agencies require two weeks of lead time to recruit participants for your study. That clock starts once they can begin preparing for your recruit, so it doesn’t include all the back and forth you may have to do to work out a quote beforehand.
If you need research participants in under two weeks, you may have to pay an additional fee to rush your recruit. This can quickly put a strain on your research budget and limit how quickly you can recruit participants.
How long does in-house research recruiting take?
So how much quicker is it to recruit on your own with a tool like User Interviews? The median time to your first qualified participant is only 3 hours with User Interviews. That means you could find a participant that fits your recruitment criteria the same day you launch your study.
Savannah pointed out that this seriously reduces her anxiety while recruiting for a study. With User Interviews, she knows that she’ll have quality participants to choose from the morning after she launches her study. This means she can update stakeholders about the process quickly and launch research with less lead time.
2. In-house research recruitment gives you more control
To get great research results, you need to talk to the right people. Sometimes that means finding a consumer that has used one of your products but not another, a user with a very specific job title, or someone who has had a particular experience .
Finding participants who fit the criteria of your study and will be able to provide you with relevant insights is one of the most important parts of the entire research process. Are agencies really up to the task?
How does the user research recruitment process work with an agency?
When you recruit participants with an agency, you usually work with a coordinator to manage your recruit. This means that you have to go through someone to manage aspects of your recruit.
Typically, recruitment agencies will share a spreadsheet with participant information. What's included in this sheet can vary from agency to agency, but at the most complete it can include screener survey responses. If you don’t have access to participant’s screener survey responses, you may end up with people who don’t quite fit your target. If you do get screener survey responses, you’ll need to take some extra time to translate the agency spreadsheet into something that is useful for you and your stakeholders.
What does the in-house recruitment process look like?
With User Interviews, you can see—in real time—how many participants have applied to your study, qualified by meeting your screening criteria, and been approved for participation in your study.
From the list of qualified applicants, you can manually control who gets approved to participate and schedule a session. This means you can determine exactly who participates in your study, instead of getting a list from an agency. This can help you meet recruitment quotas, ensure a representative panel of participants, and give you confidence that you’re getting the best participants for your study.
The flexibility and control that User Interviews offers helps the Edge Browser UX Research team at Microsoft fill studies quickly and, when needed, make changes on the fly.
Hannah Nursalim, a user researcher working on the team’s Agile research program, shared how when the team realized they’d made the screener criteria too narrow for one study, they were able to edit their screener without starting from scratch:
“With another recruiting tool, we’d have to remake the screener, contact customer success to redistribute, and such. Other platforms don’t have the same flexibility.”
3. User research recruitment agencies are expensive
Research budgets can be tight. Spending your budget wisely can help you make the most of each research project. This of course begs the question:How much does it cost to recruit participants through an agency versus. in-house?
How much do user research recruitment agencies cost?
On average, research agencies charge $100 to $150 per participant recruited. This is called a “recruit fee”, and you’re generally on the hook for it even if you get a bad participant in your study. For example, if you wanted to talk to people who use Facebook every day and one of the participants who made it into your study only uses Facebook once a month, you still have to pay for that person's recruit fee.
In addition to per-participant costs many agencies also charge an admin fee. This fee covers the coordinator who works with you during your recruitment process and other operating costs. Based on her experience, Lily said that this fee can be up to $3,000—or higher if you need a speedy recruit.
At the end of the day, with all the costs and fees, you can end up spending way more than you bargained for by going through an agency.
How much does it cost to recruit participants with a recruiting tool?
When you manage research recruiting in-house, you can seriously reduce your costs. With User Interviews, the recruit fee is between $40 to $80 per participant, depending on your criteria. (Plus, we’ll throw in 3 free participants on your first project, so you can save even more!)
And with User Interviews, there’s no admin fee, no surprises, and you get full control over your recruitment process.
Plus, we’re constantly pruning our participant pool, weeding out participants who lie on screener surveys or consistently don’t show up for research. In the rare instance that an unscrupulous participant makes it into your recruit, rest assured: You’ll never pay the recruit fee for a participant who lied on the screener survey.
4. Handling recruiting in-house makes it easier to communicate with research participants
Last minute rescheduling, verifying information about participants, and making sure everything is in order for your research project requires excellent communication with your participants.
💼 Need B2B participants for a study? Consult these best practices to ensure a smooth experience for both you and them.
Communicating with participants through a research agency means going through your research coordinator, which can be slow, inefficient, and impersonal. Handling recruiting in-house lets you communicate with participants directly.
Communicating through a user research recruitment agency
Many research recruitment agencies won’t allow you to directly communicate with participants unless it’s an emergency. You’ll need to reach out to your agency coordinator and have them reschedule the participant or relay any important information that comes up. This means you can’t troubleshoot before a session—or even if something goes wrong during one.
During the talk, Lily shared a story to illustrate how this can make research more difficult in practice. She was working with an agency and had a scheduled session with a participant in India. She woke up at 6:30am to attend the early session at a convenient time for the participant. But ten minutes after the session was supposed to start, there was still no participant in sight.
The agency hadn’t provided Lily with the participant’s contact information, so she couldn’t reach out to them to see if they had technical difficulties, needed to reschedule the session, or had just forgotten. Lily emailed the agency coordinator to see what was going on and contact the participant—but since it was early in the morning for the coordinator, they weren’t available.
By the end of the session time, Lily still hadn’t been able to get in touch with the participant or the agency. She’d woken up early for nothing! And she still had to pay the recruit fee for the participant.
Communicating with user research participants yourself
When you manage your recruitment in-house, you can directly communicate with all the participants you recruit. With User Interviews, you can communicate with participants through our messaging system, forwards messages directly to participants on their email or their phone (whichever they opted for), so they’ll never miss a message about an upcoming session.
This helps reduce no shows, allows you to troubleshoot with participants before sessions, and makes it easy to ask participants to sign important documents you may need ahead of time. Savannah shared an example about how important this communication can be:
When Savannah was working on a study about Intuit, she forgot to include a question about Credit Karma accounts on her screener survey (oops!). With User Interviews, she was able to ask the participants she had already approved the additional question, and get the right participants for her study.
5. You’re in charge of your own schedule
Calendars change daily. Meetings get thrown on the books or rescheduled, unexpected things pop up, and research doesn’t always go as planned. Needing to update a second calendar for research can be cumbersome and lead to double booking. Your research schedule should be flexible and work with your normal calendar to make your day-to-day a little less stressful.
Scheduling user research sessions through a recruiting agency
When you recruit participants through a research agency, they may ask for blocks of available time instead of scheduling the individual sessions around your schedule. This means you’ll have to set aside a large chunk of time specifically for research sessions, even though the sessions themselves may only take up a fraction of that time.
This makes it difficult to use your own panel to supplement your study, or even use multiple agencies at once to schedule your sessions. And if anything changes and you can no longer conduct sessions in that block, you’ll need to coordinate with the agency to make any changes and (potentially pay cancellation fees).
Scheduling user research sessions yourself
Being in control of your own schedule gives you more flexibility. With User Interviews, your calendar is directly synced with the platform—you can even link multiple calendars to make sure that everyone involved is available for the session. This can be especially handy if you’re inviting stakeholders to attend research sessions, since you won’t have to go back and forth to find a time that works for them.
You can also add buffers between time slots, so you won’t get stuck in back-to-back sessions and will have some time to decompress. Plus, any scheduled sessions will be automatically added to your calendar, so you have real-time updates about your research schedule.
Closing thoughts
Recruiting participants for research through an agency may feel familiar and therefore simpler—but traditional does not always mean better.
Managing your participant recruitment in-house is easier than ever. With tools like User Interviews, you can recruit the participants you need (however specific your criteria may be)communicate directly, schedule sessions easily, and save some money in the process.
.jpg)
