All posts
UX Research Topics
Podcasts
Field Guide
SUBSCRIBE TO OUR NEWSLETTER
UX researchers are human—and no human is immune to unconscious bias. Here’s how to minimize bias for better (more ethical) outcomes.
Bias is a four-letter word in UX research.
As a researcher, you do your best to avoid bias, to de-center your perspective and see past your preexisting judgements and opinions.
But as a researcher, you’re also human—and humans are not immune to unconscious bias.
So instead of asking, “how do I eliminate bias in my research?”, the better question might be: “how do I reduce bias in my research?”
In this article, we’ll discuss:
Research cannot be 100% objective—that is, researchers cannot be 100% objective.
As humans, we all carry personal biases, assumptions, and associations. These cognitive biases aren’t always inherently harmful; they allow our brains to reduce cognitive load for more efficient decision-making.
“We aren't impartial. We never are. We can do our best to be fair.” –Zoe Glas in 🎙️Navigating Gender, Religion, and Politics in UX Research
When UX researchers aren’t aware of their own biases (or in denial about the extent to which their personal biases can affect their work), this can negatively influence the study design, leading to unethical data collection methods, skewed analysis, and inaccurate conclusions. Researcher bias can be especially prevalent in qualitative research, because the data collected is more subjective and reliant on the judgment of the researcher.
Case in point: 6 design failures that could’ve been avoided with inclusive UX research.
Although there are steps you can take to limit the negative effects of bias in your research (and I’ll talk about those steps in the next section), most people struggle to identify and mitigate all of their biases all of the time.
“That’s why the first rule of product design is simple—be open-minded. It’s essential to be aware of your own biases and be ready to fight them.” - Nick Babich in Adobe XD Ideas
Additionally, bias is one of the most common concerns with research democratization. People who do research (PwDRs, or people outside of the core research team, like product managers, marketers, and designers) usually aren’t as well-trained to identify and reduce bias as professional UX researchers are.
This doesn’t make successful democratization unattainable. It just means that, in democratized models, UX research leaders with a professionally-tuned eye for spotting bias need to have oversight into the research, analysis, and synthesis process on other teams.
⚠️ Fraudulent or poor-fit participants can introduce bias into your study. Learn more in Research Participant Credibility: 8 Ways to Spot Bad Fits or Dishonesty.
You can't avoid bias entirely, but you can identify and reduce bias in UX research. Below, you’ll find practical strategies for keeping bias to a minimum:
You first have to know what biases exist in order to recognize and mitigate them. There are many different types of bias to be aware of in UX research, including:
This isn’t an exhaustive list of all the biases that exist (to see the broad scope of biases, check out this cognitive bias index) but these are the ones you’re most likely to encounter in UX research.
News flash: Everyone is a little bit biased.
That doesn’t mean we’re all bad people, or that none of us can behave in ways that uphold standards of ethics, fairness, and equity. It just means that in order to make fair, unbiased judgments, we need to recognize the internal biases we carry and try to work around them.
Becoming more self-aware is no easy feat, but there are frameworks and exercises that can help you work on it.
For example, the American Academy of Family Physicians (AAFP) uses the acronym IMPLICIT to provide strategies for identifying your personal biases:
As a researcher, you can add some of these steps, such as introspection, mindfulness, and perspective-taking, into your research process by listing your assumptions at the start of every project (see #4 for more on this).
Another common strategy for developing awareness of your own biases is by practicing reflexivity. Learn more about reflexivity in qualitative research.
Once you’ve become familiar with identifying bias in yourself, learn to recognize it in your teammates and stakeholders too—and don’t be afraid to (tactfully) call it out when it comes up.
Common signs of bias in UX research include:
By identifying your biases in the early stages of your research project, you can design the study to mitigate them and continue to address them throughout the user research process.
For example, Jennifer Ibrahim has a process for identifying her assumptions at the start of every project:
“I use the “5W1h” methodology to break down my assumptions about:
- who I believe my target users are
- what problems my solution solves for them
- when they would use my product (during a commute, drive, swim, etc. )
- where they would use it (car, mobile phone, desktop, watch)
- how they would use it ( on an app, web interface, physical product etc.).”
Critical friends are people who can offer different viewpoints, ask difficult questions, and encourage a more nuanced research approach and analysis. They can be other researchers, stakeholders, partners, or PwDRs from other teams.
Identify (formally or informally) a critical friend or friends, and ask them to offer their perspectives on:
By getting different perspectives on your research plan, you’re more likely to identify and reduce areas of bias throughout the course of the study.
Mixed methods research can reveal more nuanced insights, marrying the “what” of quantitative research with the “why” of qualitative research.
Only focusing on one method over the other can sometimes create bias. As Erika Hall says on the Awkward Silences podcast:
“It is too easy to run a survey. That is why surveys are so dangerous. They are so easy to create and so easy to distribute, and the results are so easy to tally. And our poor human brains are such that information that is easier for us to process and comprehend feels more true. This is our cognitive bias. This ease makes survey results feel true and valid, no matter how false and misleading. And that ease is hard to argue with.
It’s much much harder to write a good survey than to conduct good qualitative user research. Given a decently representative research participant, you could sit down, shut up, turn on the recorder, and get good data just by letting them talk. … But if you write bad survey questions, you get bad data at scale with no chance of recovery.”
The impact of this “bad data at scale” can be mitigated, in part, by pairing quantitative methods like surveys with qualitative methods like user interviews. The broader and more varied your data, the more likely you’ll be to notice inconsistencies and break through assumptions for real insights.
One of the most common bias-inducers in UX research is the recruitment audience.
If you’re only sampling from a small segment of users; failing to recruit for diversity, disability, and accessibility; or recruiting customers when you should be recruiting external participants (or vice versa), then you’re probably introducing bias into your study inadvertently.
Make sure you’re following recruitment best practices, like:
📚 Related reading: 8 Uncomplicated Customer Recruitment Strategies for Product Managers, UX Designers, and Marketers
Moderating back-to-back interviews all day is not a sustainable practice for any researcher—and the stress of that researcher fatigue increases the likelihood that you’ll make biased decisions. In her YouX 2023 session, Wellness and UX: Going Beyond the User Experience, Dr. Christelle Ngnoumen said:
“Stress is the biggest bias generator we have to contend with in our work.”
Participants are more likely to disengage and experience panel fatigue when they notice you’re also burnt out, especially if they feel like they are not being understood or heard with the right listening skills and attention. As you’re heading into research sessions, take a moment to gauge your stress levels and recenter yourself so as not to influence the tone and outcome of the session.
You can recenter yourself with a short mindfulness meditation, a few deep breaths, a quick journaling exercise, stretching or yoga, or any other strategies that work for you.
💜 ✨ Visit the 2023 Self-Care Playbook for UX Researchers to learn research-backed strategies for self-care and workplace wellness.
According to Lorie Whitaker of Rackspace, asking leading or biased questions is the biggest mistake people make when approaching user research:
“I’ve seen countless product managers ask the equivalent of ‘We’ve been working on this feature for months. It’s supposed to help you do your task. What do you think about it?’ I’ve also seen people assume they know their users because some research was done years ago and perhaps they go and visit a few customers each year and make faulty assumptions based on that and then, because time, money and resources have been invested, they refuse to challenge or validate their hypothesis.”
These kinds of leading questions encourage unnatural responses from participants who are instinctively inclined to validate your ideas, as Marie Prokopets of FYI explains:
“It’s worth repeating: you’ll get skewed results by asking questions like, “What do you think of this idea?” People want to reaffirm what you’re doing. But just because they think the idea is cool doesn’t mean it’s something they’ll put down money for, nor does it mean that there’s enough of a market for your solution.”
The lesson for researchers: Be careful of your wording. Watch out for different kinds of biased questions, like:
Your body language can introduce bias into live research sessions. If you’re pleased, surprised, or disappointed by any responses you get from participants, you must not show it.
Physical gestures and facial expressions can indicate your emotions to participants and make them behave unnaturally. For example, if you frown and shake your head when they move to click a particular button, they may choose to click a different button instead.
Designers, product managers, customer success managers, and other stakeholders can offer unique insight into the data you collect, allowing for more objective and well-rounded insights.
Invite them to join research sessions with you to observe and make note of the things that stick out to them. Their added points of interest can help fill in your blind spots, making objective observations where you might’ve been clouded with bias.
Plus, encouraging stakeholders to occasionally shadow research sessions can improve your company’s culture and attitudes regarding research—it’s a win-win.
Note-taking is one of the most under-rated skills in UX research.
Whether you're moderating a session or just observing, you have to take quick, clear notes that will help you speed up the analysis process later on.
Unfortunately, note-taking is another area of research where bias is often inadvertently introduced. It’s important to only include facts and observations, rather than trying to interpret participant behavior in your notes.
For example, if a participant frowns when they open up a product’s redesigned interface, you shouldn’t jump to the conclusion that they don’t like it. Simply write down what they did: “the participant opens the new UI, sighs, and frowns.” If you’re wondering why they responded in that way, you can always ask follow-up questions to get them to unpack their thoughts.
✍️ Looking to step up your note-taking game? Achieve faster, easier note-taking for UX research with these skillful tips, templates, and examples.
It’s impossible to eliminate bias entirely in UX research—but you can identify, minimize, and work with bias for better (more ethical) outcomes.
And a great place to start is with your recruitment audience.
User Interviews’s Recruit is the fastest, easiest way to recruit high-quality participants for UX research. Our participant scoring system evaluates researcher feedback and past participation to bring you only the highest-quality participants—and we never make you pay for no shows or sessions that didn’t work out.
It’s free to get started. Sign up today or visit our pricing page for more information.
Product Education Manager
Marketer, writer, poet. Lizzy likes hiking, people-watching, thrift shopping, learning and sharing ideas. Her happiest memory is sitting on the shore of Lake Champlain in the summer of 2020, eating a clementine.