Speed is not always the answer

‍

Joe Munko, the Director of User Research, Mixed Reality/AI at Microsoft, argues that if you’re not going to do research right, you really shouldn’t bother doing it at all. 

When time constraints force us to drop the rigor and process that incorporates customer feedback, the user research you conduct loses its validity and ultimately its value. 

Of course, teams often need to move quickly—speed is a competitive advantage that few companies can afford to ignore. But the way to deliver new (and valuable) products and features faster is not to simply blitz through research every time you need insights; instead, Joe explains, the key to faster development is timeless research:  

Timeless research is really about building long-term organizational knowledge and curating what you’ve already learned.

Disposable research is the stuff you throw away, after you ship. To be truly lean, get rid of that wasteful process. Instead, focus your research team’s time on making connections between past insights, then reusing and remixing them in new contexts. 

Sara Belt, the Senior Director of Growth Insights at Spotify, agreed:

Routine research is an optimized practice. The thing is, though, that much of impactful research is not routine and does take time.

Instead of asking how we can further speed up research itself, the question becomes how we can better integrate research into the product development practice and speed up organizations’ ability to learn and iterate overall. 

When to do quick user research

We’re with Joe and Sarah on this—infinitely speeding up the research process is not the answer. To continually develop great products faster and get them to market sooner, companies need to do a better job of curating and actually learning from insights gleaned during thoughtful research.

That is, of course, easier said than done. 

It also doesn’t solve the problem many researchers face right now, which is that companies are demanding more insights ASAP. And you know what? They aren’t always wrong. Quality insights should be easier to gather at scale. The fact that it takes so long to gather qualitative insights means that many teams aren’t able to integrate research into their decision making and product development. Our CEO Basel Fakhoury explains that:

For qualitative insights to be valuable, companies need to be able to discover and process them quickly. If it takes too long to get those insights they’re useless—companies will make their decisions in the interim without them. They’ll guess. Many times, they’ll guess wrong.

Yes, companies need to do a better job of integrating research into the product development process and building on the knowledge gained from research, instead of rushing to gather disposable insights for every project. 

But faster research does not have to mean less valid research. With the right tools and right approach, user researchers can gather quality insights more quickly and build upon that knowledge over time—gradually optimizing the entire development and go-to-market processes.

6 ways to do faster user research, without sacrificing validity 

Below, we’ve highlighted six strategies for speeding up your user research practice without losing the validity and value of your insights. 

1. Include stakeholders early and often

Stakeholders are those people who have a stake in the success of your research and in the end result. Successful UX design depends on stakeholder involvement—in order to do your job as a researcher well, you need to know what stakeholders know, and you need to know what they want or need to know from your research. 

We recommend doing stakeholder interviews at the beginning of every project. Back when he was at SoFi, we spoke with Zach Lamm—now Lead User Experience Researcher at NerdWallet—about his process for stakeholder interviews. He highlighted three key questions you can use in your stakeholder interviews to get to the heart of the problem you’re trying to solve: 

• What motivated the stakeholder to come to research with this question in mind? 
• How does answering this question fit into the broader context of the business?
• Above all, what problem are we trying to solve for users?
  

These questions will help you suss out what successful research will look like for everyone involved.

📖 UX Research Field Guide: Internal Stakeholder Interviews

Involving stakeholders at the beginning of research is essential for the success of that project. Involving them often is essential for creating a culture of research within your company, and ensuring that the insights you gather become part of long-term organizational knowledge. 

Immerse stakeholders in the research process by allowing them to observe and participate in research. For example, you could have stakeholders:

• Be active (but non-disruptive) observers—write down notes and questions for later
• Be notetakers for a research session
• Ask questions in a live chat, which a moderator can choose to relay to participants
• Summarize observations to present to non-researchers
• Translate insights into actionable recommendations for their departments

2. Ask the right questions

Disposable research is the stuff you only use once, in a single context. It’s like those gross paper straws that get all soggy after the first sip—there’s not much value after that first use. (Seriously though, they make eco-friendly straws out of metal and glass and even bamboo these days. The wet paper thing has to stop!)

To avoid ending up with single-use insights that will get thrown out after your current project is finished, make sure you’re asking the right questions. Jakob Nielsen explains how the wrong questions can lead users to give you replies that—while they may answer your direct question— don’t actually give you information about their real preferences:

Whenever you do ask users for their opinions, watch out for the query effect: People can make up an opinion about anything, and they'll do so if asked. You can thus get users to comment at great length about something that doesn't matter, and which they wouldn't have given a second thought to if left to their own devices.

In other words, the wrong question will give you data—but that data won’t tell you much about who your users really are, what they truly value, or what they actually think of your product (which is kind of the whole point of doing research in the first place). 

Leisa Reicht, Head of Research and Insights at Atlassian, explains that the query effect often comes into play when research is too tightly framed around a single product or feature. When research is too zoomed in, researchers risk leading the user to opine on things that they never would have been concerned about otherwise, creating false positive results.

To avoid this outcome, you should:

A) Make sure the scope of your questions is specific but not too narrow. 

Say you’re doing a usability study to get feedback on a new top nav’s ease of use. Instead of honing in on that part of the page and asking about the navigation directly, let users first click around and provide general feedback, including (but not only) about the navigation. 

You can gently nudge users to talk about the navigation eventually, but don't put blinders on from the get-go. Not only will you avoid influencing their feedback, but the insights you gather about the rest of the page can also be put to use in future projects.

B) Avoid asking leading questions. 

For example, don’t ask users if they think the loading experience on your app is too long. If you do, you’ll inevitably get data that says you should speed up loading times; while that’s probably not a bad idea, it might not deserve to be a top priority—other improvements might have a bigger impact on the actual user experience.

Instead of asking users about the loading experience directly, take note of any unprompted mention of loading times, animations, etc—if users bring it up themselves, you know it’s something worth considering.

3. Automate the participant recruitment process

For many companies, fast access to quality participants is the biggest hurdle preventing researchers from getting qualitative insights when they need them. 

A study by NN/g found that it takes an average of 1.15 work hours to recruit just one participant for research—and that time goes up more if there are specialized requirements. The same study found that the average no-show rate for research studies was 10.6%, which means for many studies you’ll need to recruit 1 or more extra participants to be safe. 

Put another way, for a study with five people, you’d spend almost an entire workday recruiting. Multiply that by however many studies and participants your team needs to do quality research, and the cost of research recruiting (both in terms of money and time) can quickly add up. 

If you want to speed up your user research, your recruitment process is a good place to start. Here’s our own lightweight solution for (semi-)automating participant recruiting with just a few tools:

User Interviews

User Interviews makes it easier to recruit quality research participants, fast—whether through our database of over 400,000 users or your own panel. If you choose to tap into our participant pool, the time it takes us to qualify the first participant for a study is 2 hours on average.

Naturally, we use our own product to source participants for all of our studies. We filter our own participants by customer segments + when they last participated/were invited to avoid oversampling the same users again and again. User Interviews also allows us to automatically generate Zoom links, collect signatures on NDAs and consent forms, create screener surveys, and sync calendars–which is really useful for scheduling sessions with multiple people (we typically have a moderator and notetaker present).

Get started with User Interviews—your first 3 participants are on us!

Slack + Zapier

We use a Zapier integration that identifies “confirmed session” emails and pushes a Slack notification to our company-wide research channel. Team members use emoji replies to volunteer as a moderator (🙋‍♀️) or notetaker (📝) for that session. We coordinate details between the moderator and notetaker in threaded comments on the original message. Adding this step helps our research go more smoothly. And since we already pay for Zapier and Slack, this extra efficiency doesn’t cost us anything. 

4. Do more remote research

Researchers did a lot of remote research in 2020, much of it out of necessity. But many of the UX researchers we spoke to said that of all the changes they had to make in response to COVID-19, remote research was the thing they plan to keep doing more of in the future.

Unmoderated user research is especially well-suited for remote—and there are plenty of tools out there to help you do usability testing, surveys, card sorting, etc. while saving time, money, and effort. Since users complete these studies on their own time, researchers are able to spend more time designing studies, analyzing data, and presenting results to the team.

📖 The Best Video Conferencing Tools for Remote User Testing | Your Remote User Interview Toolkit

Remote research is also easier to recruit for—participants can be located pretty much anywhere, and often just need a laptop to take part in the research.

Behzod Sirjani, Head of Research and Analytics Operations at Slack, shared his advice for why and how to conduct remote research: 

👂Further listening: Running Amazing Remote UX Research Sessions with Sonya Badigian 

5. Curate your insights — and reuse them

We’ve already talked about why you should stay away from disposable insights and instead focus on doing research that adds to your understanding of users’ needs, pains, and points of view. But how do you do that? How do you continuously build on a body of knowledge, instead of just adding more research artifacts to the pile?

Caitria O’Neill, Senior UX Researcher at Google, also suggests parsing out evergreen nuggets of insight from study-specific findings.

Usually when I'm doing a deck, I'll separate out the information that's evergreen, so anything about the user's behaviors, motivations, barriers to changing, current method of solving a problem.

And then also, I'll dig into their mental models in there—what they expect, why, what are their current app uses, the barriers and differentiation between those scenarios where you might want to use one or another—that kind of stuff. I want to be able to strip off the end of any deck and combine those together at the end and say, this is what you need to know about the user to design a Blue Sky version.

Properly storing and curating research results in a repository or library is essential for being able to recall evergreen insights and reuse good research. You can create something simple and searchable in a shared tool, or get more sophisticated, like Microsoft’s Human Insights System. Whatever system you use, it should be one that people can easily access in order to make use of the insights stored.

Nielsen Norman Group recommends that a research repository have the following attributes for ease of use:

• Supporting tags and metadata 
• Searchable by keyword, project, team, etc.
• Hosted in a tool that people can easily access, use, and learn
• Portable, so that repository elements can be easily exported
  

6. Invest in Research Ops

When researchers are asked to deliver insights in a short amount of time, they often struggle—and the quality of their research suffers.

That’s because insights don’t just fall out of the sky—there are dozens of steps involved in the research process, each with its own considerations and challenges. When researchers have to handle everything themselves, it’s difficult to execute every step of the research process with the level excellence and attention it deserves, much less quickly. 

Many UX-savvy companies are adding a Research Ops layer to take some of the operations burden off of researchers, freeing them up to focus on what they do best: research! A Research Ops function can help your UXRs do all of things we mentioned above, as well as handle participant communications, manage tools and platforms, train and onboard others to perform research, and so on. 

📖 Research Ops: What It Is, Why It's So Important, and How to Get Started

Fast research ≠ low-quality research

User researchers have come up with myriad ways to reduce the time it takes to do research, from Rapid Iterative Testing and Evaluation (RITE), guerilla testing, design sprints, dedicated rapid research teams, to an ever-growing stack of tools designed to automate different parts of the research process. “Used thoughtfully,” says Sara Belt, “all of these strategies speed up research without compromising the integrity of it.”

But if you’ve accelerated your research practice to the point where the insights you’re gathering lack integrity or are single-use only, you’re going too fast. And ironically, this can end up slowing things down in the long run; instead of accumulating quality insights and expanding organizational knowledge over time, you’re forcing your team to revisit similar problems over and over again.

Long story short: Moving fast is important—and it can feel like no matter how fast you’re going, there’s always another company passing you on your left. But speed doing research with quality and care should never lose out to doing research quickly. 

Instead of trying to race through the research process, the goal should be to pair thoughtful efficiencies in places like research recruiting with efforts to integrate research into the product development process and create a system for continually expanding long-term knowledge. 

After all, you can only shave off so much time from a single study before it becomes meaningless. But if you improve your organization’s ability to learn from research, in time you’ll be able to arrive at better decisions, faster. 

Is participant recruiting slowing down your research? Give User Interviews a whirl and start quickly recruiting quality participants with Recruit, or manage your own panel with Research Hub.