Note to the reader:
This part of the field guide comes from our 2019 version of the UX Research Field Guide. Updated content for this chapter is coming soon!
Want to know when it's released?
Subscribe to our newsletter!
Knowing when to use which type of research is an essential instinct for UX researchers to hone. In the last chapter, we gave an overview of the different types of user research methods. In this chapter, weâre going to take a closer look at qualitative and quantitative research, in particular.
Why? Because combining qualitative and quantitative UX research methods is the most effective way to create a comprehensive portrait of your customersâ wants and needs.Â
Properly acquired and analyzed, the high-volume of data in quantitative research helps you uncover what is happeningâtrends, issues, and opportunitiesâand prove hypotheses.Â
Qualitative research, on the other hand, adds a layer of humanity to user data, delivering details that add depth and a deeper understanding of not only whatâs happening, but why itâs happening.Â
The trick to choosing the right user research method is knowing what kind of data you need to answer your research question. In this chapter, weâll go over the differences between qualitative and quantitative UX research methods, the pros and cons of each one, and how they complement each other in a mixed methods approach.Â
â
The primary difference between qualitative and quantitative UX research lies in the nature of the dataâwhich impacts how it is collected and then analyzed.
In qualitative research, researchers gain insight through direct observation of a small group of people, often engaging in follow-up questions that dig deeper into the âwhyâ behind certain attitudes and behaviors. Qualitative research can also be adapted on the fly as researchers adjust study protocols based on participant engagement and response.Â
Common qualitative research methods include interviews, focus groups, field studies, qualitative usability tests, and co-design sessions.
Quantitative research, on the other hand, usually involves collecting large quantities of numerical data from a much larger group of people, often indirectly through surveys or analytics tools. Insights are derived using mathematical analysis of the data to quantify a problem by answering the questions, âHow much?â and âHow many?â Because statistical significance is a key goal of qualitative research, testing protocols are rigid and not subject to change over the course of a study.Â
Common quantitative research methods include user tests, surveys, click tests, card sorts, and A/B tests.
â
â
Whatever kind of product youâre working on, and wherever you are in the product development cycle, itâs important to know which kind of UX research will get you the answers you need.Â
â
We love qualitative UX research at User Interviews. Qualitative methods can provide valuable insight when you are trying to:
The use cases for qualitative research stretch across the product development and design cycleâwhen youâre brainstorming, in discovery, validating a concept, testing the usability and desirability of a finished product, preparing a go-to-market strategy, iterating post launch, doing a redesign⊠every step of the way.
Qualitative research brings the human element to bear on whatever question youâre exploring. It lets you hear what people think in their own words. It helps you compile detailed information that is both valuable in itself and also a great jumping off point for additional research directed at solving specific challenges.Â
â
Stakeholders love hard data. Compiling data to drive a strategic research agenda with management is far from the only reason to engage in quantitative UX research, but it is often the catalyst for major research efforts.Â
Quantitative research can help you to:
It can also lay the foundation for (and help you make the case for) doing the qualitative research you need to find solutions to problems.Â
Quantitative UX research is most commonly applied when you already have a working product and are trying to evaluate its usability. It can also be very effective at uncovering the answers to broad, high-level questions through statistical analysis to either validate or disprove your hypothesis.Â
â
Qualitative research methodsâwhich may be behavioral or attitudinal, moderated or unmoderated, remote or in-personâcan be further categorized into five different types.Â
Some kinds of qualitative research (like grounded theory) are infrequently utilized in a UX research context. Theyâre more commonly used by researchers in the fields of sociology, anthropology, and other social sciences from which they originated. But thatâs not to say you couldnât use grounded theory for UX research, if a problem called for it.Â
â
In ethnographic research methods, researchers observe user behavior in participantsâ natural environment. This approach is a powerful way to understand context, observe real-world product use, and gain deep cultural insights about participantsâ day-to-day lives.Â
Humans are wired for story, which is why narrative research often packs an outsized punch. In this approach, researchers run in-depth interviews with a very small number of participants in order to create a cohesive narrative that reveals themes and patterns.
Phenomenology uses a combination of interviews, observation, documentation, video recordings, etc. to help researchers understand the what, how, and why behind a particular phenomenon or event. This method helps describe and interpret lived experiences, which in turn helps uncover participantsâ perceptions and motivations.
While phenomenology aims to describe an event, grounded theory aims to explain why an event happens, by uncovering the social and psychological processes behind it. This is typically done through a combination of user interviews with 20 to 60 participants and in-depth document research.
The purpose of the case study is to relate, in detail, a real example of a specific type of experience. The nature of a case study can be either explanatory or exploratory, but the aim in either case is to gain access to deep understanding of how things happen in the real world.Â
â
As with qualitative research, quantitative methods can be broken down into categoriesâwhich type of quantitative approach is most appropriate depends on the nature of the issue you are researching, the type of information you are after, and the study protocols involved.Â
The following decision tree illustrates the process:
â
Descriptive research uses a wide variety of methods to identify the characteristics and frequency of a study topic while also looking at associated trends and categories. It relies on observation and measurement to deliver insights into the what, where, when and how something happens.
Correlational research looks at how two or more variables that are similar and interdependent relate to each other. This type of research uses mathematical analysis to show how each variable affects the other. Results are often presented using diagrams or statistics. Â
This type of research looks at the cause-and-effect relationship between two variablesâone dependent and one independentâthat are not related to each other.
Used to verify an argument, experimental research takes a theoretical approach that focuses on a theory and then helps researchers identify whether a given statement is right or wrong.Â
â
Quality researchâwhether itâs qualitative or quantitativeârequires quality participants. But the challenges of recruiting are different depending on which research methods youâre using.
â
In quantitative research, you start with a large âpopulationââthe entire group you want to studyâand then you create a âsample,â which is a subset of the specific individuals who will participate in the study. This process is often called âsampling.â For this to work, you need two things: 1) a large sample size, and 2) a solid sampling design that guarantees a random sample, which is critical to ensure truly accurate results.Â
â
In qualitative research (especially for B2B or other highly targeted studies) the key is finding the perfect participantsâpeople who fit your customer profile exactly in terms of not only demographics and geographics, but also psychographics, behavior, and specific criteria relevant to the study parameters. Recruiting participants for qualitative studies involves non-random sampling and appropriate screening to deliver the best results.
â
User Interviews can help with both types of recruiting, but weâre especially great at helping researchers fill qualitative studies that require vetted, good-fit participants who meet the specific criteria of your study.
Once youâve collected your data, what next?Â
Regardless of whether your data is qualitative or quantitative, it isnât much use to anyone until itâs been analyzed and synthesized. The methods you use to analyze your data will depend on the methods you used to gather itâand unsurprisingly, thereâs quite a lot of difference between quantitative and qualitative research analysis.
Weâll be covering user research data analysis in more detail in a later chapter. But hereâs what you need to know in a nutshell.
â
Qualitative data to yield a wealth of information, but not all of it is relevant to your research goals. Qualitative analysis involves sifting through the raw data to find patterns, themes, and stories that tell you something meaningful about the product, the user, or both.
Quantitative analysis is comparatively (deceptively) straightforward. It involves a lot of number crunchingâbut what you are doing at heart is trying to understand how people use a certain product, what problems they may be experiencing, and where improvements could be made by looking for patterns in the data.
â
â
We know thereâs a lot of information in this chapter, so before we wrap up with a brief discussion of mixed methods, let's take a moment to summarize what weâve learned about each type of research with a list of pros and cons.
â
â
Oftentimes, the best way to answer your research question with both certainty and nuance is to take a mixed methods approach.Â
Mixed methods user research is exactly what it sounds likeâthe practice of using both kinds of research methods in a single study. Side note: If youâre blending multiple qualitative methods (and you often will) or multiple quantitative methodsâbut not the two togetherâthatâs referred to as hybrid research. Similar, but different.
Many UX researchers opt for a mixed method approach. Thatâs because quantitative and qualitative methods are actually highly complementary. Quantitative research helps you identify specific problems by measuring the âwhatâ and the âhowâ, and provides hard data that can be quickly analyzed and understood. Meanwhile, qualitative data helps you uncover the âwhyâ behind an issue or opportunity. Each type of data helps fill in the gaps left by the other, giving you a holistic picture of exactly whatâs happening, why itâs happening, and how you should address it.
Mixing methods also helps you avoid making false assumptions that can cause you to travel down the wrong path, wasting time and resources on developing a solution that users donât really need.Â
For example, if quantitative research shows that a particular step in your conversion journey takes twice as long as any other part of the process, you might assume that that step is simply more time-consuming. However, if you then run a qualitative test to observe how people do that step and collect direct feedback about the experience, you may learn that the instructions are confusing and users are getting hung up on trying to interpret them.Â
The key to successful UX research is knowing which methods will give you the answers you need, and how they can be combined with other methods to give you the most complete and accurate picture possible.Â
Which is an excellent segway into the next chapter, How to Choose a User Research Method, which is all about the frameworks user researchers use to decide on the specific methodologies their research question requires.