Of course, qualitative research doesn’t stop being valuable as soon as you have sufficient data to run a t-test. It’s enlightening to watch a user researcher interview a user and realize just how differently they use the product than we do. At Facebook, where I worked prior to User Interviews, some of the best product ideas came from observing the workarounds users created when the product lacked some functionality. For example, researchers learned that many people would screenshot posts in their feed so they could return to them at a more convenient time, spurring Facebook to build more tools for saving content and finding it again later. These insights might never have emerged if the company hadn’t been talking to its users.

UX research can also support analytics teams, and vice versa. A/B testing is a common method to run experiments and determine how different versions of your product perform on certain metrics. At Microsoft, we had a rule of thumb that about a third of your experiments will go as you’d expect, a third will go the opposite way, and the remaining will be inconclusive and require further tests. Additional user research is often necessary for the latter two scenarios to understand why the data moved the way it did. But it’s also important even in the scenario where your gut was correct, because you want to make sure there aren’t further improvements you can make that you might not have thought of.

For example, at Facebook, I worked on a product called the “Local Bookmark,” where we’d suggest new events and restaurants to people.

We’d test different recommendation algorithms, like “Trending Restaurants” or “Places Your Friends Like.” The results would often surprise us, and our user researcher would dig in a bit more. For example, we learned that people don’t automatically trust their friends’ tastes—rather, they trust experts, or a specific set of friends whom they believe make the best suggestions.

Data scientists should also support the user researchers on their team. One of the most important parts of product analytics is looking into trends that might not yet be on anybody’s radar. What actions do your most frequent visitors take? Do new users use one part of the site or app more frequently? What features are being used more over time? And, after discovering some of these trends, the analyst should pass the learnings on to a researcher, so that they can actually talk to some of those users and better understand why these behaviors are emerging.

Right after I started at User Interviews, I ran a regression analysis to better understand how researchers decide whether to continue using our products or not. As part of this project, I produced a list of companies that had used us frequently before abruptly stopping; a teammate and I conducted interviews with a few of them to better understand why. As it turned out, the reason didn’t have much to do with User Interviews directly— many of these researchers had simply switched jobs, and nobody at their previous company had picked up the baton. This led us to brainstorm ways to better support more people in a company beyond just the person who initially signed up. For instance, we launched a reminder email for new folks to create an account when a colleague invites them to join a project:

Today, the biggest companies in the world use a blend of user research and data science to learn everything they can about their customers and their market. It’s not an either/or proposition; rather, both disciplines are essential and can continuously strengthen each other. As a data scientist, I understand the value of unlocking insights about thousands and thousands of customers at once. But it cannot replace the surprising results that emerge when you talk to those customers one-on-one.