If you’re reading this, chances are you know that UX research is important. Whether you’re an experienced moderator, a product manager who plays researcher from time to time, or someone new to the field, we hope this guide has a little something for everyone. Here, we’ll focus on the UX research process as we’ll describe it throughout the field guide. Think of it as a framework for infusing research throughout the product development cycle and embedding user research throughout your organization. Your product development cycle might look a bit different. No matter. Details aren’t critical here and the framework is flexible.
We’ve broken different research methods, skills, and approaches into three key product development stages where they are often applied. Of course you might use any of these in other stages, and that’s fine too.
When launching a research project or sprint, work backwards. Why are we doing this research? What do the internal stakeholders need to learn to move the product forward? What information would be actionable? A good rule of thumb is to research as early and as often as possible. This will save you from wasting time building prototypes for solutions no one needs. From there, working within time and budget constraints, you can build a research plan that includes:
In this guide we’ll cover all of the above, with lots of detail on typical methods used in user research. Here’s how that breaks down.
This is where you’ll learn about the project’s goals internally, where you’ll grow to understand the market for the product, its potential users, and ultimately product requirements to meet the market need. In short, you’ll broadly define what product you’re building and why. You’ll decide to build the “right product” here, without knowing exactly what it will look like.
A great product that doesn’t solve a market need is not a great product. “Being early is the same as being wrong.” Build the right product at the right time. This is absolutely critical for your product’s success, for your business’s success. Don’t go by instinct, by HiPPOs, by what’s worked in the past, or by a bunch of quant that doesn’t provide context for the why behind the what. When in doubt, talk to people, and five is usually enough.
Researchers need toolkits to pull from on a case by case basis. No two projects are the same, so developing your arsenal of techniques empowers you to handle any situation. Here are some common methods researchers use in the discovery phase.
Across the below methods, researchers are often analyzing participant data on a few different levels.
Observation: what is the person I’m talking to (ideally watching) do? Body language, unsaid behavior are often as important as what is said.
Understanding: What mental models does this person employ? What is the why or context behind their behavior and preferences?
Analytics: Looking at aggregate data, what patterns do we see and what solutions do these suggest?
Understanding what “the decider” (the real person in charge of the product development cycle) ultimately values and needs to know is not only politically smart, it’ll help you do better work. A big part of the researcher’s goal is to align and articulate both business and user goals, and find that precious intersection of both. Don’t skimp on the time necessary to understand your key stakeholders’ goals. Bring your best user interviewing skills to the process.
Ethnography is all about observing people in the context of their actual lives, their natural habitats. You can understand how they behave in groups, how they interact with your product outside of a lab environment. It can be hugely beneficial in considering use cases, problems, and solutions you might not have otherwise. What people do is so often different than what they report, and ethnographic studies can be a great way of getting around this issue.
Ethnography is one type of field study. Field studies more broadly encompass any research where you're going to your participants instead of having them come to your lab, or speaking with them remotely. Field studies provide real life context.
When you need to understand long-term user behavior, diary studies are a great option. If you are short on budget to run field/ethnographic studies, diary studies can be a solid alternative. You’ll be able to understand habits, changes over time, motivations, and long-term customer journeys. This can help you both build a better product, and better understand user behavior over the long-haul after your product has launched.
Focus groups are not exactly the, eh hem, darling of UX research circles these days, but they certainly have their place in market research environments. When you are very early in developing, and marketing, your product in particular, this type of research can be useful in getting a broad view into the audience you seek to serve. You can then augment this data with other methods for better context and insight into motivations. You can also observe a lot of people relatively quickly. Market research may encompass a variety of these methods, but generally is an overall, often initial, pass at understanding the market you wish to serve.
In generative research you’re looking to find solutions to a defined problem, or a hypothesis of what that problem might be. Generally you’ll talk to participants one on one, asking them a set of non leading questions, with an emphasis on past behavior, testing out your problem and solution hypothesis. You might confirm your thinking or find wildly different conclusions. That’s why this type of research can be so valuable early on. You’re looking for patterns across interviews to make informed decisions.
With more than 17,000 employees and multiple business units around the world, Adobe cannot afford to overlook user research as part of its core organizational strategy. But the process of incorporating research into everyday workflows can be a challenge—central research teams and vendors are often spread thin with many responsibilities.
One of the ways that Adobe tackles this challenge is to democratize its research program across the entire organization. Teams use a mix of techniques that fit into their everyday workflows.
Adobe encourages engineers to integrate usability research into their daily lives. One technique that developers use is to take turns answering questions on live chat. Each developer on one product team spends 30-60 minutes per week answering support questions from customers. On the other side, customers are unaware that they are speaking to an engineer. Typically, customer support teams or success functions manage real-time conversations with customers.
The recruiting process is clear—identify existing customers who are running into roadblocks.
With this information in-hand, Adobe’s developers can identify themes around pain points and prioritize feature releases accordingly. The research process takes very little time out of each engineer’s work time each week.
At this point, you have a good sense of your market, its needs and pains, and the kind of solution you can offer. You’ve taken those ideas and built some wireframes, high fidelity mockups, or interactive prototypes. (Thank you Balsamiq, Sketch, and InVision). Now you need to understand if these designs help users solve their problems, how they interact with them, and where they get hung up.
If understanding what product to build in the first place is the most important aspect of research, understanding if you’ve succeeding in building the right product is certainly the runner up. The more variety you can present in your prototypes the better. Try a few out of the box ideas next to what feels “safe.” Truly, you never know how users will react unless you test. And that’s why this stage is so very important. You may even be missing key features you haven't thought of. Once you have your analysis in, you can refine your prototypes and re-test until you’ve found a solution you’re happy with.
This is the most popular type of study we see at User Interviews, and for good reason. In qualitative usability testing you’re getting feedback on a tangible prototype or live product that you can immediately use and act on. Taking the time to talk to just 5 people or so and ask them to “think aloud” as they interact with your prototype can provide an incredible return in terms of time and money spent to insights derived. You’ll have the opportunity to evaluate implicit and explicit cues, and find patterns quickly, rapidly improving your prototype to bring it to market (or improve the existing version) as fast as possible.
Task analysis helps you better understand your user’s goals, and how they go about achieving them. This can be coupled with a variety of other methods, and is an important aspect of evaluating the overall user experience for most apps.
Once you have a live, working version of your product or feature, you may wish to do more granular ongoing testing. You can a/b test, a/b/c test, single variant test or multivariate test. However you construct your experiments, the goal is to get quantitative data that shows which version best achieves the goal of the test, generally some hybrid of a business and user goal, for instance conversion rate on an ecommerce site.
Long an afterthought in product design, accessibility is getting more and more attention in UX circles, and rightfully so. As with so many systems and institutions, a lot of unconscious bias is built into many designs, and accessibility testing is a great defense against these pervasive issues in our society. Whether it’s designing for special needs, to be inclusive across racial, geographic, and gender lines, or to build a product that can span the socio-economic conditions in which an audience may use it, accessibility should never be an afterthought in your product research.
This is just what it sounds like. As part of a task analysis, usability testing, or other study, this is a popular way to assess an app, wireframe or prototype’s user experience. You record the first click someone makes to accomplish their goal. When you build user journeys for your product, that first click can determine a lot in terms of your user’s ability to complete their task in as direct and efficient a way as possible. You’re looking to optimize your website or app’s navigation in the process.
Your product probably has a bunch of information. You need to organize that information in a way that helps users accomplish their goals. That’s what information architecture (IA) is all about, and card sorts are a great way of testing how you categorize and label content. There are open card sorts where your users define the groups themselves, then place topics into them. There are closed card sorts where you define the groups and users place the topics into them. Card sorts can provide great insight into your user’s mental models and how you should organize your content.
In a tree test, you’re coming to users with a completed list of categories, any subcategories, and the labeling for all. Users then try to find certain information within the tree. You provide the information they should find (the task) and the tree (the categorization you’re evaluating). Their ease in completing tasks based on the structure you present is how you measure success and adapt your system as needed. It is often used following a card sort to help build useful and intuitive navigation systems.
Depending on your business, you may have some strong direct or indirect competitors. When you plan to launch a new product or feature, taking the time to have users evaluate your experience against your competitors can be invaluable in making sure you’ve created something at least at parity with, if not exceeding the current capabilities of, your competitors’ offerings. Is this feature going to be a key differentiator for you company? It better be better than the competition.
After you’ve launched a product or feature, there are a variety of ways to keep the conversation going with users and prospective users alive. Surveys, web analytics, and feedback channels are just a few of the popular options for this kind of research. You’ll coordinate with a variety of customer facing and analytics minded teams here to get the data you need to understand how your product is performing in real life.
If you’re lucky and smart, you’ve been testing throughout the previous stages of product development. You have a good sense of who your users are, and the context they live in when using your product. You know which solutions will best solve their problems and empower them to complete tasks they value. You’ve built those solutions and they are live. You are done!
You are not done. There are a few reasons for this. First, if you’ve focused on qualitative data to date—good for you, we love qualitative data—you have not thoroughly tested your actual real life experience in the wild with a multitude of quantitative data, including data that ties closely to your business metrics. You’ll want to make sure what you put out there is accomplishing the goals you set for it in the first place. Second, things change. What worked well a year ago or more may no longer be the best solution. You may get hints of this through ad hoc user feedback, NPS scores, proactive surveys, or other channels. You’ll want to adapt. Finally, you have put the very best solution of what you thought to build out there, but it’s certainly possible you haven’t found the best solution yet.
Depending on the importance of any given feature’s role in your business or user journey, you’ll want to continue testing and optimizing some features until you hit a point of diminishing returns. Others are less critical and good enough may very well be good enough. In any case, set it and forget it is not a smart product strategy. Ongoing listening methods help keep your product useful, impactful, and relevant over the long-haul.
User intercepts, sourced participants, short surveys—like NPS—longer surveys targeted at a particular aspect of the user experience; there are a variety of survey types and tools to help you meet your ongoing listening goals. Things to consider include: Where will you get the most actionable insight to improve your product or experience now? Who is contacting your users across the company, asking for feedback? Is it too much or just enough and can you coordinate to optimize these touchpoints?
Analytics can be a treasure trove of great quantitative data if you know how to analyze it and connect insight to action. Whoever is managing analysis and this kind of technology at your company—perhaps it is distributed or a central resource—make friends with them, or become them, so when problems or opportunities arise, you can adapt your product and experience accordingly. Keeping a close track on key user flows and business metrics will help you see changes over time. Quickly observing sudden variances that might suggest a large bug or problem will help keep your product in tip top shape for years to come.
Bugs come up and need to be fixed. That’s just a true thing. Make sure you have good bug reporting and systems to address them rapidly. Your dev team is probably on top of this, but you can use this data to understand where current or historical frustrations may have impacted the user experience in ways you wouldn’t have known otherwise. FAQ and support desk reporting can similarly help you understand where users are getting caught up or confused in the product experience. Taking ongoing steps to clear up these moments in the product itself will improve the user experience and make your support team oh so happy.
Still with me? Wow, you’re awesome! So we’ve covered a lot about the kinds of methods you’ll use in your research, and I hope you’ll sign up to receive the entire course in your email inbox, which will also cover tools, logistics, recruiting, and more. Or just sign up for the modules that interest you.
Before we part, a final word on actually making all this happen. Knowing what to do and how to technically do it is a huge part of the battle. But actually making it happen in a real organization can yield a whole new set of challenges. Your job here is to continually sell and prove the idea that research is indispensable to the product development process at every stage. We have a full module coming on this topic, but if you aren’t already doing research all the time, find someone willing to listen to you, start small, do your homework, share your insights widely.
In addition to highlighting the value research has added—finding or tweaking the perfect prototype, building the right product in the first place—make sure to highlight the mistakes that would have been made without research. Without this step, research can be ignored, be taken as unseen and hence an unimportant aspect of product development. Often its value is in avoiding disaster (or at least unnecessary missteps). Share these moments too. Once you start building the story that research is imperative within and outside product teams, start doing more of it, sharing it more widely, perhaps even training other teams to do it.
You’ve got this!