Note to the reader:
This part of the field guide comes from our 2019 version of the UX Research Field Guide. Updated content for this chapter is coming soon!
Want to know when it's released?
Subscribe to our newsletter!
Knowing when to use which type of research is an essential instinct for UX researchers to hone. In the last chapter, we gave an overview of the different types of user research methods. In this chapter, we’re going to take a closer look at qualitative and quantitative research, in particular.
Why? Because combining qualitative and quantitative UX research methods is the most effective way to create a comprehensive portrait of your customers’ wants and needs.
Properly acquired and analyzed, the high-volume of data in quantitative research helps you uncover what is happening—trends, issues, and opportunities—and prove hypotheses.
Qualitative research, on the other hand, adds a layer of humanity to user data, delivering details that add depth and a deeper understanding of not only what’s happening, but why it’s happening.
The trick to choosing the right user research method is knowing what kind of data you need to answer your research question. In this chapter, we’ll go over the differences between qualitative and quantitative UX research methods, the pros and cons of each one, and how they complement each other in a mixed methods approach.
The primary difference between qualitative and quantitative UX research lies in the nature of the data—which impacts how it is collected and then analyzed.
In qualitative research, researchers gain insight through direct observation of a small group of people, often engaging in follow-up questions that dig deeper into the “why” behind certain attitudes and behaviors. Qualitative research can also be adapted on the fly as researchers adjust study protocols based on participant engagement and response.
Common qualitative research methods include interviews, focus groups, field studies, qualitative usability tests, and co-design sessions.
Quantitative research, on the other hand, usually involves collecting large quantities of numerical data from a much larger group of people, often indirectly through surveys or analytics tools. Insights are derived using mathematical analysis of the data to quantify a problem by answering the questions, “How much?” and “How many?” Because statistical significance is a key goal of qualitative research, testing protocols are rigid and not subject to change over the course of a study.
Common quantitative research methods include user tests, surveys, click tests, card sorts, and A/B tests.
Whatever kind of product you’re working on, and wherever you are in the product development cycle, it’s important to know which kind of UX research will get you the answers you need.
We love qualitative UX research at User Interviews. Qualitative methods can provide valuable insight when you are trying to:
The use cases for qualitative research stretch across the product development and design cycle—when you’re brainstorming, in discovery, validating a concept, testing the usability and desirability of a finished product, preparing a go-to-market strategy, iterating post launch, doing a redesign… every step of the way.
Qualitative research brings the human element to bear on whatever question you’re exploring. It lets you hear what people think in their own words. It helps you compile detailed information that is both valuable in itself and also a great jumping off point for additional research directed at solving specific challenges.
Stakeholders love hard data. Compiling data to drive a strategic research agenda with management is far from the only reason to engage in quantitative UX research, but it is often the catalyst for major research efforts.
Quantitative research can help you to:
It can also lay the foundation for (and help you make the case for) doing the qualitative research you need to find solutions to problems.
Quantitative UX research is most commonly applied when you already have a working product and are trying to evaluate its usability. It can also be very effective at uncovering the answers to broad, high-level questions through statistical analysis to either validate or disprove your hypothesis.
Qualitative research methods—which may be behavioral or attitudinal, moderated or unmoderated, remote or in-person—can be further categorized into five different types.
Some kinds of qualitative research (like grounded theory) are infrequently utilized in a UX research context. They’re more commonly used by researchers in the fields of sociology, anthropology, and other social sciences from which they originated. But that’s not to say you couldn’t use grounded theory for UX research, if a problem called for it.
In ethnographic research methods, researchers observe user behavior in participants’ natural environment. This approach is a powerful way to understand context, observe real-world product use, and gain deep cultural insights about participants’ day-to-day lives.
Humans are wired for story, which is why narrative research often packs an outsized punch. In this approach, researchers run in-depth interviews with a very small number of participants in order to create a cohesive narrative that reveals themes and patterns.
Phenomenology uses a combination of interviews, observation, documentation, video recordings, etc. to help researchers understand the what, how, and why behind a particular phenomenon or event. This method helps describe and interpret lived experiences, which in turn helps uncover participants’ perceptions and motivations.
While phenomenology aims to describe an event, grounded theory aims to explain why an event happens, by uncovering the social and psychological processes behind it. This is typically done through a combination of user interviews with 20 to 60 participants and in-depth document research.
The purpose of the case study is to relate, in detail, a real example of a specific type of experience. The nature of a case study can be either explanatory or exploratory, but the aim in either case is to gain access to deep understanding of how things happen in the real world.
As with qualitative research, quantitative methods can be broken down into categories—which type of quantitative approach is most appropriate depends on the nature of the issue you are researching, the type of information you are after, and the study protocols involved.
The following decision tree illustrates the process:
Descriptive research uses a wide variety of methods to identify the characteristics and frequency of a study topic while also looking at associated trends and categories. It relies on observation and measurement to deliver insights into the what, where, when and how something happens.
Correlational research looks at how two or more variables that are similar and interdependent relate to each other. This type of research uses mathematical analysis to show how each variable affects the other. Results are often presented using diagrams or statistics.
This type of research looks at the cause-and-effect relationship between two variables—one dependent and one independent—that are not related to each other.
Used to verify an argument, experimental research takes a theoretical approach that focuses on a theory and then helps researchers identify whether a given statement is right or wrong.
Quality research—whether it’s qualitative or quantitative—requires quality participants. But the challenges of recruiting are different depending on which research methods you’re using.
In quantitative research, you start with a large ‘population’—the entire group you want to study—and then you create a ‘sample,’ which is a subset of the specific individuals who will participate in the study. This process is often called ‘sampling.’ For this to work, you need two things: 1) a large sample size, and 2) a solid sampling design that guarantees a random sample, which is critical to ensure truly accurate results.
In qualitative research (especially for B2B or other highly targeted studies) the key is finding the perfect participants—people who fit your customer profile exactly in terms of not only demographics and geographics, but also psychographics, behavior, and specific criteria relevant to the study parameters. Recruiting participants for qualitative studies involves non-random sampling and appropriate screening to deliver the best results.
User Interviews can help with both types of recruiting, but we’re especially great at helping researchers fill qualitative studies that require vetted, good-fit participants who meet the specific criteria of your study.
Once you’ve collected your data, what next?
Regardless of whether your data is qualitative or quantitative, it isn’t much use to anyone until it’s been analyzed and synthesized. The methods you use to analyze your data will depend on the methods you used to gather it—and unsurprisingly, there’s quite a lot of difference between quantitative and qualitative research analysis.
We’ll be covering user research data analysis in more detail in a later chapter. But here’s what you need to know in a nutshell.
Qualitative data to yield a wealth of information, but not all of it is relevant to your research goals. Qualitative analysis involves sifting through the raw data to find patterns, themes, and stories that tell you something meaningful about the product, the user, or both.
Quantitative analysis is comparatively (deceptively) straightforward. It involves a lot of number crunching—but what you are doing at heart is trying to understand how people use a certain product, what problems they may be experiencing, and where improvements could be made by looking for patterns in the data.
We know there’s a lot of information in this chapter, so before we wrap up with a brief discussion of mixed methods, let's take a moment to summarize what we’ve learned about each type of research with a list of pros and cons.
Oftentimes, the best way to answer your research question with both certainty and nuance is to take a mixed methods approach.
Mixed methods user research is exactly what it sounds like—the practice of using both kinds of research methods in a single study. Side note: If you’re blending multiple qualitative methods (and you often will) or multiple quantitative methods—but not the two together—that’s referred to as hybrid research. Similar, but different.
Many UX researchers opt for a mixed method approach. That’s because quantitative and qualitative methods are actually highly complementary. Quantitative research helps you identify specific problems by measuring the ‘what’ and the ‘how’, and provides hard data that can be quickly analyzed and understood. Meanwhile, qualitative data helps you uncover the ‘why’ behind an issue or opportunity. Each type of data helps fill in the gaps left by the other, giving you a holistic picture of exactly what’s happening, why it’s happening, and how you should address it.
Mixing methods also helps you avoid making false assumptions that can cause you to travel down the wrong path, wasting time and resources on developing a solution that users don’t really need.
For example, if quantitative research shows that a particular step in your conversion journey takes twice as long as any other part of the process, you might assume that that step is simply more time-consuming. However, if you then run a qualitative test to observe how people do that step and collect direct feedback about the experience, you may learn that the instructions are confusing and users are getting hung up on trying to interpret them.
The key to successful UX research is knowing which methods will give you the answers you need, and how they can be combined with other methods to give you the most complete and accurate picture possible.
Which is an excellent segway into the next chapter, How to Choose a User Research Method, which is all about the frameworks user researchers use to decide on the specific methodologies their research question requires.
