100% of the research we’re supporting these days is remote, and we know many of you are either new to remote research, or could use some tips to get the most out of it. We were fortunate to have Behzod Sirjani, Head of Research and Analytics Operations at Slack join us in conversation with our VP of Product, JH Forster to talk about doing your best research remotely.
The recording below has chapters to help you navigate to the part of the conversation that’s most interesting to you, captions to help you follow along, and links to the tools and resources mentioned in real time.
To ensure you have the most information you can about this important topic, we’re also including this writeup, with expanded content and links to the tools and resources Behzod and JH mentioned in the webinar.
Here’s what we’ll cover:
The benefits of remote research
Things to consider in a COVID-19 world
Best practices for remote research
Basic research practices, remotely
Tools to help you do remote research
Examples of creative remote research
Closing thoughts
The benefits of remote research
First things first, since your research is likely to be remote for the near future, let’s talk about what remote brings to the table. Sure, it’s a change, and there are certain benefits that in-person research is uniquely able to provide, but remote has a lot of upside!
💻 90% of user researchers said they've worked exclusively remotely since the pandemic began. Read the stats on remote work amid COVID-19, the rise of remote user research tools, and more in ✨The State of User Research 2021 Report.✨
Efficiency and cost
Every research project has a cost (both money and time), and remote research is no different. Remote does, however, offer you a chance to be more efficient in terms of both money and time. Your participants can save time and money by simply logging into the session from wherever they are, instead of driving to a testing location. Because of this, we found that you can also pay slightly lower incentives for remote sessions than in-person ones.
Based on an analysis of over 25,000 research sessions conducted on the User Interviews platform, we made a recommendation of $80/hr for remote sessions, while the incentive we recommend for in-person sessions is $100/hr. If you’d like to see what incentive we’d recommend for your study specifically, you can use our Research Incentive Calculator to get a unique recommendation.
Participant diversity and inclusion
Remote research allows you to talk to anyone, anywhere. This is a huge advantage over in-person studies, which limit your participant pools to whoever is in your area and can afford to take the time to drive to you to complete a session. Hopping on a quick research call on a lunch break is a much easier ask, and enables you to talk to many more people.
We’ve heard many researchers sing the praises of remote for connecting them with participants they never would have otherwise. In the early days of our podcast, we heard from Cat Anderson, UX Writer at AP Intego, about how much she loved the geographic diversity remote studies offered.
I wanna throw down and say two enthusiastic thumbs up for remote user testing. I think that it is a fantastic way to access populations that you would never have access to. I mean, we're on the east coast here. I don't just wanna talk to east coast people. When I have a day where I can talk to Mark, the guy who sells janitorial supplies in the Ozarks, that is my best day. When would I ever, ever get a chance to talk to that guy under other circumstances? Only because of this awesome job, do I get to talk to that guy and learn about his life and learn about his pain points, when purchasing business insurance for his janitorial supplies company. And then, my job is to take what I've learned from that guy and help him, and help it be better for him. That to me, is just the best thing.
Behzod also highlighted the ways remote research allows you to connect with more diverse populations in terms of ability as well. He highlighted a tool called Fable, which allows researchers to connect with populations of participants with different abilities to improve the accessibility of their product.
📖 Read more: 6 Unexpected Ways COVID-19 Has Changed UX
Conducting research in the participant’s own environment
Remote research allows you to connect with participants in their own environment, which may have different environmental factors than your usability lab. Distractions from notifications on their personal laptop, crying kids in the background, wonky internet, or anything else that may arise in their normal life gives you a chance to learn more about how your product might actually work for them.
Behzod highlighted this in the webinar, pointing out that many people’s work environments have shifted due to COVID-19.
One of the things that you're going to really spend time thinking about if you work in enterprise or in consumer is just how different is it to be using these tools in different environments? And instead of looking at that as a challenge, because they're not in their office, embrace that and use that moment where they are hyper aware of those differences.
Dig in and say, Hey, look, if you're normally using this product in an office, right now, you're not in an office. You're not around your teammates, you don't have maybe super fast internet or whatever else is great about being in an office. How does this do? How does it work differently for you now given where you are and what's going on.
Even in this uncertain time, people still need to get their work done, and if they need your product to do it, it’s important to understand how the changes in their environment affect the way they interact with your product.
Things to consider in a COVID-19 world
Everything is different right now, and things change day to day, but when it comes to research, Behzod said a little empathy can go a long way.
Things are different and difficult for everyone right now
Everyone is going through something uniquely difficult right now. Whether it’s dealing with caring for your children while working, anxiety over the state of the world right now, difficulties getting work done at home, something else, or a combination of all those things, everyone’s dealing with difficult stuff right now. Take that into consideration during your research sessions, and be understanding of speed bumps that may happen.
📖 Read more: How COVID-19 Has Changed the Way UX Researchers Work
Be kind, considerate, and prepared
Being a researcher requires a lot of empathy, and that skill is more important now than ever. Behzod shared some great advice on opening up and creating a safe space for research to occur in this uncertain time:
I think for you as a researcher, you need to be more patient and think about the emotional arc that you normally go through in a study, what it looks like to build rapport and to get people comfortable. And you probably have to stretch that out a bit. Spend some time understanding where they're at, what it's like being at home. Figure out how to open up and be vulnerable with them.
So then you can create a safe space. And if you realize that actually they're not in a place to participate in research, just be human with them for the amount of time that you've had in the study. Maybe all that you can do for them is sit, listen, and be kind. That'll be enough.
Best practices for remote research
Remote research, just like any research, has a few best practices. Behzod highlighted three things to be aware of, especially for those new to remote research.
All the regular research rules apply
All research should have a purpose, remote or not. Before launching your research study, think about why you’re doing this research, what you need to learn from it, and how your team will use what you learn to move forward. Behzod dove deeper into this in his talk at the Strive 2019 conference, Don’t Start with Data.
Conversations may be initially different
With technology comes technical difficulties. Remote sessions may begin with “Are you there?”, “I can’t hear you”, or “Can you turn your video on?”. While this may feel a little uncomfortable at first, just be patient and understand it’s a part of the process.
Our VP of Product, John-Henry Forster, suggests having scripts on hand for when these situations occur. Clear instructions around how to share your screen, video, and audio can help participants feel more comfortable with the environment and get you through technical difficulties quickly. You can share some of these instructions ahead of time, so prepared participants can work out technical difficulties beforehand, and have them on hand during the session.
Because technical difficulties can happen, Behzod suggests leaving a little extra time for sessions. So sessions that would have been 30 minutes may become 45 minutes. If everything goes smoothly, you get extra time with your participant or time to process your notes.
Behzod also suggests using the first few minutes of a remote session to get a feel for the latency of the call. Because the connection will be different every time, it’s helpful to understand how much latency there is on a call early in the conversation. During the webinar, Behzod detailed how he assesses the delay on a remote call.
I actually right now can kind of tell how good John-Henry and I are able to talk because we were talking before this call started. So I know sort of what the delay is, but often very early on, I try to determine how much delay there is between me and the other person, so that as I'm pausing, I can account for that in my pauses and it feels less like I'm talking over them, which is something I try to be really attuned to.
Over-plan and over-communicate
Planning and communicating are especially important for remote sessions. Behzod and John-Henry both recommended preparing and communicating a little more than you may be used to.
Behzod always has a Slack channel open with his teammates during a session. This makes it easy to touch base if anything goes wrong or if anyone needs to change anything. Behzod advises researchers to brief any team members who will be attending remote research on what they can and can’t do, and what they should expect. This makes it easier for everyone to participate in a constructive way. He also has a backup plan in case something happens, like his internet going down before a session.
John-Henry also encouraged researchers to keep whatever line of communication they have with their participants open during the session. Whether it’s email, a messenger, or a phone number, it’s important that participants can get in touch with you at the beginning of a session if anything has gone wrong.
During the session, you don’t get a lot of the body language cues you can get in-person during remote research. This makes over-communicating key to getting your point across and making your participant feel comfortable. Behzod outlined how he does it:
I think a lot of people assume that you can nudge people that you're talking to in the same ways that you do in person, but you have way less like stimuli coming through and way less nonverbal communication. So I try to spend more time up front saying. You know, I'm really here to just get your perspective. I'm looking for your feedback, and I will often repeat things that they're saying.
I will double back on stuff and make sure that I'm really over communicating. So like, “Hey, John, I heard that you say this, did you mean this? Or can you dive into that a little bit more?” Because I want to make sure that they know, in every way I can communicate, that I'm here for them and I'm excited for the work that we're doing. And that it’s a valuable use of my time and hopefully their time. I think a lot of people just under communicate, which is, it's a natural thing to do because you're used to doing in person.
Basic research practices, remotely
We saw a lot of questions during the webinar about how to do some research functions remotely. How do you recruit the best participants? How do you take useful notes? Behzod and JH dove into these in the webinar, but we’ll elaborate a bit more in this section.
Recruiting participants for remote studies
Recruiting participants for remote or in-person studies isn’t that different. Once you’ve settled on your research question, you can start to narrow down who you need to talk to to answer it. If you need help finding participants, we can give you 3 free participant credits to get started on User Interviews.
Here’s what Behzod had to say about recruiting participants:
The harder question I think is how to find those people...For some of the more cold calls, just being clear about why it is you're reaching out to people. I think a lot of people feel like research requests are spam because they're often not targeted or not specific. I think the better thing you can do is start small and sort of iterate towards a message that resonates.
So, “Hey, Brenda, I'm reaching out from Slack because I noticed that you're on a bunch of different workspaces. And I'm actually really interested in understanding that kind of cognitive overload of switching back and forth. Do you have half an hour for us to chat on the phone?” It’s very pointed, even if we've never talked before. I think it's helpful for me to explain why I wanted to reach out to you.
Having a specific ask and reassuring the participant that they have something unique to offer your study means they’re more likely to reply to your request. Need a little bit more guidance? Here’s our outline of how to write great participant invites.
JH also recommended asking participants an open ended question in your screener survey. If you’re sourcing from a big pool of participants, an open ended question can help you get an idea of how detailed someone’s feedback may be.
If you’re looking for more guidance on recruiting great participants, check out these resources:
Jan Chipchase, The Field Study Handbook. Behzod recommended this book for understanding how participants think about being involved in research and ensuring you’re recruiting the right audience.
A General Guide to Recruiting the Right Participants is a general guide to recruiting participants for your study. Start here if this is your first time recruiting participants for a study!
Why Participants Participate in User Research is an interview with our Participant Marketing Lead about what motivates participants and how researchers can best reach out to them.
Using Facebook to Recruit Participants is a guide to using marketing strategies to recruit participants for research. We outlined our process for using paid Facebook ads to fill our participant pool with the best participants out there.
Taking notes for remote research
Having a note-taker during your research is a great way to expose others in your organization to research and get some help with documenting your sessions. But if you can’t get someone else to jump on a session and help you take notes, Behzod and JH had some tips for better note-taking.
Behzod recommended splitting your screen and taking notes that highlighted key words and time stamps.
I've done this different ways and I think that you should develop a note taking solution that works best for you. I do think split screening is really helpful.
One of the things that I typically did in terms of note taking was not writing everything, but I would write a key word, and then the time. And then I would go back to that part of the interview. So, I used to write “juicy quote” and the time for things that I thought were really fascinating and interesting.
...There are different tools. I think Zoom itself lets you transcribe recorded sessions. There are other tools that do that. So if one of your biggest concerns is transcription and being able to search through that so you don't have to take as many notes or notes in the same way, I'd encourage you to check that out.
JH said he found a loose note template helpful for taking notes during a session. Here’s one that uses a table format to help you keep everything organized.
Both hosts pointed out how helpful timestamped transcription is when going back through your sessions to find the best quotes to share with stakeholders—it saves you a lot of time and effort during your research analysis.
Tools to help you do remote research
There are tons of tools out there to help you do the best remote research you can. From participant recruitment tools to fully managed solutions, there are remote research tools for all kinds of use cases and budgets.
Tools mentioned in the webinar
User Interviews - for recruiting participants remotely and managing research projects
Fable - for recruiting people with different abilities to meet accessibility guidelines
Google Slides - for collaborative testing and presentations
Figma - for designing collaboratively and creating mockups for usability tests
Dscout- for conducting remote diary studies
UserTesting- for conducting remote user testing sessions
UserZoom- for conducting remote user testing sessions
Loom- for recording screen share videos to share with participants, your team, or facilitating remote research.
Otter.ai- for transcribing sessions
Descript- for transcribing sessions
Grain- for transcribing sessions and collaborating with teammates
Other tool resources
The Best Video Conferencing Tools for Remote Research. We talked to researchers and used survey data to find the best video conferencing tool for remote research.
The UX Research Tools Map. We made a comprehensive list of UX research tools, and put them together in a subway map graphic to help you choose the best tools for your team.
Examples of creative remote research
Remote research doesn’t have to just be generative interviews or usability tests. There are tons of different ways to do research remotely. In this section, we’ll cover some of the methods people asked about during the webinar, and some creative ways we’ve seen people do remote research.
Remote mobile user testing
While there are tools that allow users to share their mobile screens during user testing, Behzod suggested a unique method. It’s called Laptop Hugging. Basically, a user hugs their laptop with the camera facing outward while navigating on their mobile device. This allows the researcher to see what a user is doing on a mobile device, without any extra tools. If you’re looking to get creative, or for tools to do remote mobile testing, this article from UXPA is a great place to start.
Remote contextual inquiries
When an attendee asked Behzod how to conduct a contextual inquiry remotely, he encouraged them to consider if the same context still applied right now.
One question I would ask you is. Does the context still apply? For my team, one of the things that we think about is like how people work. And so some of the questions that we were asking about work environments actually have to change because the work environment has changed. And so the questions that we asked about office dynamics or the interplay between being next to someone and working online is no longer relevant.
Once you’ve settled on a contextual inquiry question that works for your team right now, Behzod suggested doing a diary study. This allows you to learn about your participants over time and gives you the opportunity to collect different kinds of data.
You can do this by using a diary study tool like dscout, but there are also other creative ways to run a diary study.
Katie Phillips, at the time a Senior UX Researcher at Australia Post, on how she ran a diary study using Slack.
Tatiana Vlahovic, Staff UX Researcher at Credit Karma, on how she ran a diary study using Google Docs while working at a previous company.
Other creative remote studies
There are tons of ways to get creative with remote research, and outlining all of them would take a whole new article. So here’s some inspiration from creative researchers who’ve run out-of-the-box studies.
The team at Optimal Workshop on running a successful remote co-designing session.
Brandie Smith, UX Researcher at Metromile, on how she asked participants to write love letters and breakup letters to learn more about the emotional impact of car insurance.
Ashlee Edwards Brinegar, Researcher at Instagram, on presenting her research findings through a podcast.
Closing thoughts
Thanks again to everyone who attended this webinar and helped shape it with your questions! We really enjoyed being able to put on this event and were overwhelmed by the response it received. If there’s anything else you’d like to learn more about during this uncertain time, please reach out to me at carrie@userinterviews.com. We’ll do our best to continue to provide content that helps you navigate your research practice, good times or bad.
Behzod was an incredible guest, and offered to answer questions we didn’t get to in the webinar via Twitter. You can reply to his tweet below if you have any questions we didn’t get to, or just want to say thanks!
.jpg)
.jpg)
%20(1).png)
