SUBSCRIBE TO OUR NEWSLETTER
Learn how to use metrics stakeholders care about to make your research impactful
Before we dive into the task of showing the value of user research, let’s cover what we mean by user research anyway. In this context, user research is any activity that involves getting feedback from your users during the product development cycle. This could mean user interviews to discover what your users want from a specific feature, usability tests to improve an existing design, or even diary studies to uncover pain points about your current process. User research can be one or all of these things.
In short, user research is valuable because it can help you speed up your learning process, saving money and time on each project. Proving this to stakeholders, however, can be a challenge, which is why many teams put off research. It takes a bit of budget at the beginning to launch user research, but it’s an investment in the project’s success as a whole.
Your team could be exceptionally passionate about producing great work, try their best to be sympathetic to the user's needs, and attempt to put themselves in the user’s shoes when they create solutions, but you won’t create the best product you can without actually talking to your users about what they need. User researcher Jared Spool has perhaps one of the best stories about how even teams that mean well can make fatal mistakes without research. It starts with a very simple form: two inputs and two buttons.
He was working on a checkout flow for an ecommerce site, and there was a login/registration form that popped up once you hit “checkout.” The in-house team Jared was working with liked the form, and thought it would enable repeat customers to purchase again more easily. When Jared went to do usability testing with customers, he found that the form frustrated them. One user even said, “I’m not here to enter into a relationship. I just want to buy something.” Plus, repeat customers couldn’t always remember they had purchased before, and would register, or had forgotten their passwords.
So Jared advised that the team take away the login/registration page in between hitting the checkout button and actually checking out. The result? A 45% increase in the number of customers purchasing, which translated to an extra $15 million the first month the change went live. Over the first year, the company brought in $300 million more in sales.
Moral of the story? Even if your team thinks they’re doing the right thing for users, you won’t know until you’ve gotten feedback from them. That extra step that the team thought was making things easier for everyone was actually costing them revenue, and lots of it.
This may seem counterintuitive, especially since doing user research at the beginning of the product development process usually takes some extra time, but knowing what you’re building and why can save you time later on. According to the Interaction Design Foundation, investing in UX and research can really make a difference later on.
“It is far less expensive to avoid designing a problematic product in the first place than to fix it later. When organizations invest in UX during a project’s concept phase, they reduce product development cycles by 33 to 50 percent. This is vital; a one-fourth delay in the amount of time it takes to bring a product to market results in a 50-percent loss of that product’s profit.”
Even though you may need to take some extra time at the beginning of a project to do user research, you’ll save time later on by knowing what your customers want and need from you. Studies have shown that developers spend 50% of their time on avoidable rework, some of which can be avoided by doing research at the beginning of a project. When developers earn an average of $36 an hour, and studies have shown that development projects with high uncertainty take considerably longer, that’s not only time wasted, but a considerable amount of money.
For example, let’s say you’re building a new website and it takes your developers 40 hours (which is a low estimate). If you make mistakes on the usability of your site, which could have been avoided through testing, your developers will spend ~20 hours fixing the problem, costing your business at least $720 extra. That’s money and time that could have been better spent on research beforehand and other projects.
Showing the value of something typically qualitative, like user research, can be difficult. You can make it easier by establishing a clear metric that your research is impacting.
Stakeholder interviews can help you learn more about what people expect from your project. How do they think about product development, research, and how it all fits together? What do they want to see happen in this project? What metrics are they being held accountable for?
Often, research gets pushed to the side with flimsy excuses like “we don’t have time for that right now” or “we already know what our customers want”. Stakeholder interviews can help you get to the bottom of why people are making those excuses, and in turn, how you can demonstrate value to your stakeholders specifically.
In order to demonstrate the concrete value of research, it’s absolutely essential to know what these stakeholders value. Learning this at the beginning of your project can help you identify the best metrics to focus on during research. It can also help you communicate value in the same terms as your stakeholders, which makes it easier for them to get on board.
How do you understand what it is that your stakeholders value in a short interview? It’s a pretty big question, but there are some ways to steer stakeholders towards communicating what they value clearly.
We spoke with Zach Lamm, Senior UX Researcher at SoFi, about the questions he asks stakeholders after receiving a research brief. He identified three that he continues to return to:
If you’re talking to stakeholders who have already given you a research brief, those questions are a wonderful foundation for a good interview. For those talking to stakeholders who are hesitant about research in the first place, we recommend getting a little more specific. If this is your first time talking to stakeholders, these questions are even more important.
These questions may seem a bit heady for an interview about a research project, but knowing how your stakeholders and the business see value is the key to showing research’s value in your organization. Knowing this at the beginning of your project can help you make more concrete ties between the values of your stakeholders and the ways your research supports those values.
For example, if a stakeholder says that the one thing that’s key to the success of your business is reducing customer churn, you can demonstrate the ways in which your research helps your team accomplish that goal.
Be sure to ask specific questions about the metrics they use to measure success and how they check those metrics. This will help you identify metrics your stakeholders are already using, and hopefully can point you in the right direction as far as data you can use to set benchmarks.
This section of the article owes a lot to Kate Rutter’s wonderful talk, Measure What Matters. In it, she outlines how to simplify the process of choosing metrics and focus on what really matters to your business’ success. We’ll be using her framework and applying it to continuously demonstrate the value of user research to your team.
To start, you’ll want to think about what the core problem your project is trying to solve for people. Hopefully your stakeholder interviews pointed you in the right direction and identified the problem you’re solving. Whether you’re building something completely new or working on a feature, think about your project in terms of what human need you’re trying to solve. Here’s how Kate Rutter lays it out:
First, you’ll need to think about what people you’re trying to solve for. To make things a little more concrete, we’ll go through this section with a specific project in mind. Let’s say we’re a website that delivers pet supplies. The team has heard complaints about how difficult it is to update payment information on our site, so they’ve decided to launch a project to fix it.
In that case our diagram would look something like this:
Take a moment and think about what this would look like for your project. It should be pretty simple and easy to understand. The point of this exercise is to recognize the core problem and how the user fits into the picture, not map out the whole project.
Now on to the difficult stuff. How do we define the metric our research will improve before we’ve built anything? Luckily, we’ve already done stakeholder interviews and we hopefully know what this project should accomplish. That’s a great starting point to identify a metric that can show the value our research brought to the project.
Let’s go back to our online pet supply store example. If we’re trying to make it easier to update payment information on our site, the metric should demonstrate that the task is easier to complete now than it was before. In this case, a good metric might be the % of people each week who were able to update their payment information in less than 30 seconds.
This metric is specific in that it defines something exact enough that it’s clear what you’re looking for. It is normalized in that it is a percentage of the whole, so even if you get a huge influx of users, the metric won’t be adversely affected. It is comparable in that it is collected weekly, so you can track your progress over time without getting lost in the data. It is actionable in that we can do something about it, by trying to push people under that 30 second mark.
Following this framework when setting metrics helps you to stay away from vanity metrics (e.g. tracking the number of people changing payment info, which likely will only go up as your user base grows). It also makes it clear what you should be looking for in a metric that conveys value definitively to your team and stakeholders.
Once you’ve done your research and measured your metric, you’ll need to present your findings to your team. This is the part of the process where everyone else on your team sees the value research can bring to the table. So let's dig in to what makes a great research report.
First, you’ll want to think about the metric you worked so hard to establish and set your work up for success with. How does it connect with your stakeholder’s main business concerns? How does it show value to the things they value? Tying your work back to these things is the key to impactful presentations that make stakeholders research believers.
So, let’s go back to our online pet store example. Let’s say your stakeholders were launching this project in the first place because they were sick of wasting customer support’s time with payment info updates. How would your research help solve that problem? You could dig in to exactly how much time customer support is wasting on payment information updates, to show a real dollar value.
Then, if the product changes have already been made, you can show that number before and after your project was completed, in tandem with your research metric. If you’ve saved enough money, you can even highlight that the cost of research is covered in what you’ve saved from making the changes.
If you have access to even more data, you can take it one step further. Let’s say one of the main business concerns of your stakeholders is reducing churn. Can making it easier to update payment information help to reduce churn? If you can find out through your analytics tools, you can compare what churn numbers are within the cohort of people who were able to change their payment information quickly vs. if they weren’t able to change their information, or couldn’t complete the task quickly.
Tying your research to key metrics and cost in this way helps stakeholders see research in terms of what it can bring to the table, instead of what it takes to complete it.
To get some tips on how to make a research presentation really stand out, we talked to Caitria O’Neill, Senior UX Researcher at Google. Caitria recommended telling a story with your research report. Not only is it easier for people to remember a story than a huge pile of data, it’s also a lot more engaging in the moment. Between what you’ve learned from your stakeholders and what you’ve learned through research, a story may have already emerged. For example, if stakeholders are really concerned about increasing sign ups and the design you’re testing shows that users aren’t signing up because of a point of friction in the process, that’s a great story to tell.
“Bullet points frame the whole thing. What are the top three sections? That's the storytelling tool. If you're doing a report on the quality of the app, you could make it fat, slow and big, or something like that. That's a storytelling device. You're really emotionally affecting people with those three section titles. I wouldn't suggest using them that bluntly unless you have a really good relationship with your team, but you can still use those same kind of interesting, provocative tools no matter the type of tool you're using.”
If your research is going to make it past the day of the presentation, you need to make sure it’s accessible and understandable to someone who finds the report later. This can also help you show the value of future research by comparing it against old metrics. Caitria suggests structuring each slide as a bit more than just “what happened”, but “what happened, why it happened, and what can be done about it.” These slides can also include metrics about what you learned during your research.
She also suggests including a cheatsheet at the end of each presentation that makes it easy to understand the study quickly. This makes research more approachable to anyone coming along later.
You can also help yourself out later by making the data, notes, and report easily findable. You can use templates like our Launch Kits to keep all your research data in one place. This can help you establish important benchmarks to show research’s value over time and make a case for research over time, not just on your first study.
By getting more data-driven with your research efforts, you can make a compelling case that user research is worth it to your team. Tie your research to key metrics stakeholders care about, communicate your findings with the team in terms of the metrics impacted, and make a compelling case for user research. Hopefully this article has given you the tools you need to communicate value to your stakeholders and earn user research a seat at the table
Carrie Boyd is a UXR content wiz, formerly at User Interviews. She loves writing, traveling, and learning new things. You can typically find her hunched over her computer with a cup of coffee the size of her face.