Transcript
[00:00:00] Jo: I think at the end of the day, research has had such a hard time getting a seat at the table and for the job, not to be diminished too, you know, you just talked to users that they fear that this will actually happen.
If we try to democratize research that people will say, oh, actually anyone can do it, it's very easy. But I think the reality of it is it will give a taste of what it really means for people to be more research centric and more user centric. And that will actually empower the teams to validate and value research much more.
[00:00:26] Erin: Hello everybody and welcome back to Awkward Silences. Today we have Jonathan Widawski . He is the founder and CEO of Maze, a tool probably many of you have used. Uh, We're so excited to have you here today, Jonathan, to talk about how to scale and democratize research within an organization. Hot topic, big topic. Good. One to dig into early in this fine new year. We see ourselves in. So, Joe, thanks for joining us.
[00:01:12] Jo: thanks for having me. Very excited to have this chat with both of you as well. I think it's been top of mind for everyone, and we've seen a lot of talking shop around what it means to democratic advocates. And I'm excited to dig in.
[00:01:24] Erin: JH is here too.
[00:01:26] JH: Yeah, hot topic. I think just personally, like we've had a lot of fun playing around with the maze tool, so it's always cool to talk to people behind the tools that we are playing around with on our side. So it's kind of a cool combo for me.
[00:01:35] Erin: All right. So let's dig in, you know, let's zoom out first because democratization does not happen in a vacuum. So what is the context in which we're seeing these trends happen? What is kind of the current state of research and, you know, what's driving this trend
[00:01:53] JH: democratization
[00:01:54] Erin: within research within an organization.
[00:01:56] Jo: Yeah, I think what we saw is that the reality of how orgs have evolved is that the research and the research department hasn't really scaled the same way that today the design and the development departments have evolved so we see that the ratio remains pretty unbalanced where you have ratios of roughly one researcher for five designers and then five to 50 for designers to developers.
And so what that ultimately translates into is that the capacity for teams to build is 50 X, the capacity for teams to learn. Because at the end of the day, your capacity to learn is limited to what your researchers can do. Right? And so we talk to a lot of people, we talk to people at Google and different companies, and what we see is that they have this kind of massive backlog that researchers have to handle.
And so it leads to them having to either cut out decisions, meaning the teams are led on their own to make those decisions or they have to overwork to get those decisions actually done. So the demonstration, I think, comes from that, it comes at first from the investment that product teams are making in building and scaling research departments.
So the question becomes, how do we solve more and more decisions that need to be made daily when the research department is not scaling at the same pace that the building let's say department scaling. I think that's where democratization comes from.
[00:03:11] Erin: and why, why aren't research teams scaling, right? So like let's not take that at face value. Like one solution here would just be to scale research teams. Is that not happening or should that be happening? Like why?
[00:03:23] Jo: Yeah, I think it should be happening. I think it will happen in the future. I mean, we've seen the same thing for design and I think that's the very interesting parallel in my opinion, as we saw how design exploded in the past five years. Right. It used to be, I mean, I come from a design background, on my end. I used to be called just a designer.
And then you see the sophistication that starts to evolve, right? Where you go from designer to UX and UI designer into more sophistication in even the nomenclature of the roles and tools also empower for this conversation to happen.
So in the past when teams were using software that were just mindful designers, like a sketch of this world, that we're not collaborative design wasn't needed in the design department. What Figma really achieved is creating conversations inside the whole organization around design. And then that led to an expansion of design inside the organization and expansion of the design roles inside the organization, because of all of this.
Do you think it was no longer a designer conversation. It was an everyone conversation. And so I think we can see the same playbook happening for research where our research right now is limited to the research department, almost a black box within the organization. How do we make sure that research conversation is something that happens everywhere in the org?
[00:04:32] JH: Okay. So it sounds like maybe two things are true here. Right? So research organizations do need to scale to support the kind of building teams, as you suggested. But we're using this kind of democratization thing as somewhat of a stop gap, maybe in between, but there's also value in it. And we think some of that's going to persist. Is that kind of like your general view of the world at the moment?
[00:04:52] Jo: Exactly. Exactly. That's a good summary. I think that at the end of the day the research department, we need to scale for it to happen, we need to build a case around research, becoming a conversation for everyone. And I think that if you look at the past ten years that happened in design, but more closely to research that also happened, for example, in data and data analysis, right?
Look at the state of data analysts 10 years ago. And you had small limited data analysts within the organization that was a small BI department and then tools like amplitude and mixpanel. They came in and they made the data conversation, something that happened everywhere, right? Because all of the sudden data self -serve for everyone, everyone would have around the data.
And that led to a growth of the BI teams inside the organization, because the value was seen and perceived by the management teams, by everyone, within the org, but also that made the conversation around data and much, much more natural within the organization. So we see that for data. We saw that for design, I think the next logical step is for it to happen in research.
[00:05:46] Erin: Yeah we talk about that a lot internally, this kind of parallel of, you know, analytics or quantitative insights and qualitative insight and going from it being the black box, the centralized, I don't have any idea how I'm going to find what this user did. I need to ask this other team, I can find it now.
And obviously what you're seeing going along with that is like data literacy, right. You know, in the past, like, well, yeah, you can access the data, but you don't know what it means. It's kind of a dangerous tool. Right. And so you're starting to see more data literacy crop up and qualitative and research will go the same way.
Why is this happening? Right. Like, you talked a little bit about. There's many more engineers. There's a huge capacity to build. But why are we seeing research become so central in organizations?
[00:06:35] Jo: Good question. I think that's what's happening. In the world of the future, that will be, there will be winners. And those winners look like the companies that build with the customers, they are the one that actually deals with the users. And so what's interesting is there's multiple studies that have been run around what is the real business value of design and what is the real business value of research?
And we sold the McKinsey piece in 2019. That was about what is the ROI of investing in design? Can we actually quantify this ROI? And what they saw was it was a very strong strategic investment to invest in design. And so, because it was proven, then we can see the shift that's happening. We can see that the organization now is investing much more inside the design and the next logical step for them to better understand the users and deliver the best product for those users is to be expanding research as well in the organization.
So I think it's just a natural occurrence of the market understanding. That building with and for your users is the best way to build products and to build software that people use.
[00:07:32] JH: And to go back to the comparisons that, you know, with data analytic tools and design tools, it feels like now it's sort of accepted that those things have been a net positive, right? By everyone having access to data in Mixpanel, amplitude, wherever it may be, or being able to comment and make things in Figma, we're seeing this collaboration and this literacy and all of these things that are positive.
But there are, you know, concerns there as well, right? Like you could make the wrong inferences from data if you're not well-trained in it, or if you're not an actual designer and you're going in there and trying to play designer, you can create bad experiences. And I think we're at a stage in research right here are those fears or those concerns pretty loudly right now, because we haven't gotten over this hump or we haven't gone through this transition.
And it's a lot like, well, there's a craft to do research and if you're not trained in that craft, you're not going to do good research. And I'm curious, what do you think about that part? Is that just like part of evolution or is there something different about research that needs to be considered as this happens?
[00:08:25] Jo: So it's interesting. I think that this has happened every time democratization as a word has come up in every space. You look at Webflow at the time and people were saying, yeah, Not everyone should be a developer and it was the same popping, but it's the same every time democratization happens.
But I think those few have come from a place where there needs to be literacy around how you run these processes. I think there's also a fear of replacement almost right. Just like developers at the time it could appear that. Webflow could replace their job with the same fear that's happening for researchers.
What does it mean to democratize research? It means that everyone can be a researcher. And I think it's very different from saying democratization of research is very different from saying replacing researchers. I think it's more of the evolution of the role that we need to embrace, which is that at the end of the day, the researchers won't be able to take on every decision that happened with the org.
And so their role has to shift from a role that's technical, running the research to a role that's educational, which is how do I help every team to run their own research. And we work very closely with Behzod Sirjani who was the head of research at slack and Facebook. And that was his role there, right? His role was entirely, how do I teach teams?
How do I help them and provide the right resources for them to be able to create great tests and create great research and analyze the data that comes out of this. So I think it's just in this role, evolution of the role is that just like the BI teams help teams understand the data for themselves the researcher role will have to evolve as well.
So, the fear is legitimate, but every time this has happened it has been proven to avoid fear because at the end of the day, it actually empowers researchers to expand within the organization. It actually makes research more visible.
[00:10:05] Erin: Yeah, you can fear the technology, but it's not, you know, going away. I, we already brought up web flow. That's where big web flow fans here at user interviews. And I think. Like I can use web flow. I think J H and I built our blog in a day, maybe two,
[00:10:21] JH: About that. Yeah.
[00:10:23] Erin: wasn't like the most
[00:10:24] Jo: Okay.
[00:10:24] Erin: blog in the entire world, but it functioned and it wasn't terrible.
And got us through a year before we got designers to help us out. But that's sort of the point is that we can use web flow. We can.
[00:10:37] Jo: Yeah.
[00:10:37] Erin: Create value with it, but someone who knows what they're doing can use it a lot better. And I think I get so amped up About tools and afraid of tools. it's going to take their job, or diminish the craft in some way that this tool dares to make it easier for me to do my job, you know, like that.
That's so scary. And you know, you also mentioned Figma and I think. What that does is it makes it easier for designers to design like good designers who can actually design. Right. But then it makes it easier for everyone else to be part of it too. And that's what we're really talking about. Right. It's not saying now everyone's a designer, but now everyone's part of the design conversation.
[00:11:16] Jo: Exactly. And Dylan Fields says these all the time, right? He always says Figma didn't make everyone a designer, but it makes design happen everywhere. And I think that's the beauty of it, right. That's really what they are not is that design becomes central to the org. So what we're trying to achieve at me and I'm, so what you're trying to achieve as well is how do we make research more central to the organization as well?
[00:11:38] JH: Yeah, no, I think what we're describing here is that it's a cool kind of counterintuitive trend, right? Where by giving this stuff away to some degree or making it more inclusive and accepting some of the risks and trade-offs that come with that, you actually make your organization more research fluent, and more bought into research.
And then in this kind of unexpected way, your role as a researcher is much more valued, right? So it's like, it doesn't maybe seem like the obvious path to get there, but you're saying that we've seen that in other disciplines and there's a good reason to believe that's going to be the trend that plays out here.
[00:12:08] Jo: Exactly. And I think that makes sense, right? Like the fear is legitimate, because I think at the end of the day, research has had such a hard time getting a seat at the table and for them not for the job, not to be diminished too, you know, you just talked to users that they fear that this will actually happen.
If we try to democratize research that people will say, oh, actually anyone can do it, it's very easy. But I think the reality of it is people. It will give a taste of what it really means for people to be more research centric and more user centric. And that will actually empower the teams to validate and value research much more.
[00:12:39] Erin: I mean, with research the proof's in the pudding, right? This is a popular topic too. Like, how are you making an impact? How are you showing you're making an impact and whoever is using research to make good decisions will have that seat at the table, whether they're a full-time researcher or not, you know, whether they're doing the research or taking advantage of someone else's research, right?
That's, who's going to I think own research and organizations, people doing and using research to make impactful decisions.
[00:13:46] Jo: Exactly. And we also come from a place where user research is very mentally loaded. When we started Maze, what we saw from customers was research that was perceived as, you know, slow and expensive. You needed to have a lab, you know, all of these things that people associate with. That was blocking them from even getting started.
So by removing some of the barrier to entry for people to actually get started with research, get a taste of research, meaning giving them the tool to just empower people in the product team to be able to run those. It creates this value that in the future, people will hire more researchers.
It will help them basically understand more of the value of a company that wouldn't have done that if they didn't have access to tools and software that would end up with value early on.
[00:14:25] JH: Yeah to come at this from maybe a little bit of a different angle, because I do think we're using some kind of analytics and design as some adjacent disciplines. You know, pull lessons out of, but those things do feel a little different than research in some ways. Right? So I'm thinking about analytics . I'm not an analyst.
I go and I spend a few hours, you know, creating a report or pulling something out. It's pretty easy for somebody who's more trained in that field to come and look at my work. Maybe spend, you know, 10, 15 minutes going through it and kind of see if it passes the smell test, right? Like, oh, you did this well, or, oh, you messed up the causation, right.
Or whatever. Similar to design, I'm going to try to play designer and make a prototype or whatever. A true designer could come look at that pretty quickly after I've spent some hours on it and be like, Hey, you miss some of these edge states or this prototype isn't set up correctly, but it doesn't take them the same amount of time to review.
Whereas research, it feels a little bit more nebulous. Like I go out and I just talk to people and I don't know what I'm doing. And I'm like, I'm doing research. It feels a little harder for the researcher to come in and assess whether or not I did a good job with it. Right. Like they almost need to like, watch that whole conversation back.
And like, was I leading? Did I go in the wrong direction? Like how do researchers do that part? Because the craft of research feels a little bit more nebulous than maybe how explicit the skill of design or analytics is. I don't know if that makes sense, but do you see
[00:15:32] Jo: No, he does. I like the angle. I think that's where the part comes in. Both providing the resources for people to ask the right questions and reviewing the script before actually going in and running these interviews. And then as you said, probably helping on how to interpret the data and run the smell test.
I think that at the end of the day, everything can be taught on how to run interviews for everyone. If you have the right teachers, which means that you still need the researchers to be able to run those smells, check, and make sure that this is working for your org.
[00:16:00] JH: Okay. Yeah. So maybe a good way to think about it is, in some of those other disciplines you can do the review at the end, like let people go off and do what they want and then somebody can come in and check it and research. It might be the opposite where it's really important to have. And review upfront before people go off and run and try to do their own thing.
Is that maybe a good way to like, think of it?
[00:16:16] Jo: Yeah, it's an excellent way. And I know, for example, again, to take the Behzod example that people can slack, what they did was they created this massive database of questions and how to ask the questions. And so they had this. Almost let's call it the research system on how to ask questions, how to not lead questions.
And that helps a lot, right? Because all of a sudden, anyone that really wanted to get involved into any form of research had the resources to actually go out and which was really empowering for the team as well.
[00:16:42] JH: Cool. And is this the type of thing you're doing at Maze? Like, I know we in the warmup, we talked a little bit about some of the testing frameworks you use in the team and stuff, like, have you found good ways to handle this approach within your own team?
[00:16:52] Jo: Yeah. So multiple teams at Maze as a product. I think what's interesting is that because we run unmoderated there are moments of collaboration before things go out live for researchers to actually review right? And because there's less actual face-to-face conversation, there's also, it's also less likely that the test would go wrong.
If someone comes in and has the capacity to review the test before it goes out. So that's what we do. We created collaboration at the moment of creation to allow for teams to review the test together before they actually send it out. And then the second thing that's interesting is that in our team, but also as a product, because anyone is empowered to run this research.
We also have to educate. And so for us that translates into multiple things. On one end, we create content on how to run this research and that people can read, but also we productize it, right? So that translates into creating templates for people to use. So that even if you don't really know how to ask the right questions, we provide you with the templates for you to ask the right questions.
And then in the most sophisticated team, what we do is we allow you to create. Right. So the researcher will come in and create this database of questions that the teams can then use to run their own unmoderated tests and research at scale.
[00:17:56] Erin: So it sounds like you've created a lot of templates and resources for folks to use. How do you get people to actually use them or train them or they need them? Cause I always think about that with any kind of internal, you know, education or enablement are people using it?
How do you get them to use it?
[00:18:13] Jo: Yeah. So we have kickoff sessions with, so we have internal researchers at maze, we have kick off sessions where the researcher will walk people through the content. And then the templates themselves are embedded into tools. Right? So for us, it's very easy because at the end of the day, we see whether it gets started from a project that you or didn't use that template, but it's actually easier to get started from a template than not so people have no incentive not to do it.
[00:18:36] Erin: Right, right, right, right, and so when new people join, do you, is this part of onboarding, right? Like, is everyone doing research or just certain departments?
[00:18:44] Jo: Specifically the product teams. So specifically the designers, the product managers, the product marketers which are the ones that need to make product decisions. So yeah, that one is it's part of the onboarding basically. So it's part of the onboarding playbook that we have that they reviewed these documents.
We actually asked for the team to read a whole blog on the as well, so that they also have context on how this is empowering our users. How other people are using it so that they get a clear picture of not only how we run it, but also how people have been running it, both our platform and other methods.
[00:19:13] Erin: I know you have different folks, you know, involved in research. You mentioned the product teams, you've got product management design, you've got product marketing. I'm curious how it's evolved to what these different teams do? Like who's doing research, who's receiving research. How are these different kinds of functions involved with what democratization looks like to me?
[00:19:33] Jo: Yeah, that's a good question. So the way we structure at Maze is that we have different pods that will own different parts of the product and they are all empowered to own their own research. So ultimately the business risk of the feature is owned by the researcher. So they will be the one actually owning what is the risk for the business.
As we build this feature, then it's passed on on the value risk for the PMs. And then finally it's passed to the designers
So generally the product managers. They take care of the value risks. So do people actually want, this is a problem that we're trying to solve and then the solution we are ideating on is the right solution for the problem we're trying to solve for. So they own both those risks. And in both cases, they will run interviews with users that we believe to be the right persona to talk to.
And they will also run unmoderated studies to validate hypotheses that they discovered through these qualitative studies that they're running. When it's passed on when the value of risk is assessed and validated, we can pass it on to the design team and the design team will own the user B2B, the feature or the thing that we're building.
And so they will be empowered to run their own unmoderated studies for testing the product. testing the design and testing the flows that they are trying to build. And then the product marketers it's interesting because we see more and more of those product marketers, both using off the thumb, but also at Maze running those tests. Testing the copy. It's a big part of the usability, but people generally do it in two steps.
So they test the copy. They stage the value proposition. They test the future's name, which is what we can get made as well for every feature that we build. So that's how we kind of cut out the different parts of the research for us.
[00:21:03] JH: Nice. You described three levels there, business risk, value, risk, usability, risk, and kind of who's responsible for each of the last two. I'm more familiar with, I'm curious, when you say business risk, what are examples of the types of questions or the things that you're trying to understand or mitigate at that stage?
[00:21:18] Jo: Yeah. So generally you don't market for what we think is a problem, right? So we'll evaluate a problem that we want to solve at an organization level. So we want to have more collaboration, right? So we'll say, okay. So how do we assess that? The tools that we have today. I Need collaboration. And so the researcher, what they will do is they will go up define different personas that we believe to be potential strong hypotheses for collaboration.
And then we will reach out to them, both qualitatively and then, through Maze, quantitatively to understand if the problem exists for them. So we'll try to stack rank the problem, basically like how big of a problem this is for you today. How would you, how much would you invest in this problem today, if you were to solve it? And that helps us understand if people actually do.
Before we get to assessing if we need to ideate a solution for
[00:22:01] Erin: It really changes the sort of tenor of the research. If you have a researcher kind of on the hook for like, I'm going to put my stamp on, this is a good business decision to do this. It is like everyone having some skin in the game, in a business. Right. Yeah. You know, you talked a little bit about like using some of the templates you've created within the, within your product and how much, guess, how much, some of your process of democratization your own research and collaborating on your research has informed how you've built maze is that happened a lot along the way.
[00:22:32] Jo: Yeah. Yeah. So we always see the limitation with what we do. You know, even collaboration. The hypothesis came from us, collaborating on treating the test and the research and we thought how can this
for before? That's how we basically created the hypothesis for it. And then qualitative studies validated these.
And that's how we can actually build collaboration. And that's been the case for a lot of these things that we can do, which is, it starts from a problem that we stumbled upon while building. And then we tried to validate that this problem actually happens for other the teams and other scale, and other industries using the product, educated on how the product evolved
[00:23:06] Erin: yeah,
[00:23:07] JH: Yeah, as I said, you, do you find that as a strength or a weakness? You're not sure. Cause I feel like we catch ourselves often on my team saying things like, you know, you know, of course we're not our user, but we sort of are. But like, you know, we use our product a lot and we are doing research. And so we have our own hypotheses based on our own usage of the product.
And sometimes I think that's like a great strength to us because it's like we identify some little rough edges that we feel strongly about and we'll just clean up. And then other times. We gotta not just run away with our own ideas here. We need to go back and go and be more methodical. So I'm curious how that plays out for you all.
[00:23:37] Jo: Yeah. Yeah. So we have a great customer experience team at Maze. And so their role is to actually get all of these ideas and feedback and try to map them to customer requests, frustration from our customers. So that's at the end of the day, we can have pen sheets on what we need to build. And then they can actually benchmark against existing data and say, okay, so what you're saying, actually no one really cares about it because no one has really complained about these things.
The thing is that sometimes for some of these features, people won't voice the problem, right? They won't actually say this is a problem for me. So you actually have to actively research for this problem to happen. So it's balancing that right. What you discover and then actually try to validate it where the customer experiences it can be kind of a barrier saying, okay, this is not a real problem.
You're just making it up for yourself. Or getting access to those insights. And that's going to educate the hypothesis we're going to solve against.
[00:24:26] JH: Yeah, good to have some counterbalances.
[00:24:28] Jo: Yeah, exactly. The check and balance of the customer experience team.
[00:24:31] JH: Yeah. Yeah. It's similar. It's like when you run an experiment on a test, it's like, we want to optimize for this outcome or metric, but we need to make sure we don't hurt these other things. Incidentally, Saturday. Yeah.
[00:24:40] Erin: I was just laughing because not that we do this, but you know, we're in the recruiting business. It's like, oh, we can go find some users with that problem. No problem. Somebody has got that problem. We'll
[00:24:48] JH: Yeah. That's also true. Yeah.
[00:24:50] Erin: We don't do that. Of course. Joe you uh, you switched, you said you were a designer and now you're leading a research company. What made you kind of expand or get excited about UX research? What are you excited about in the future?
[00:25:06] Jo: Yeah. I think that what's extremely exciting is that as we said a bit early Research is exploding. Right? What we're seeing right now is just research becoming something that every company is talking about.
I think that's what I'm excited about is that I believe that the future will be extremely user-centric right. I think companies understand more and more that they need to build with their users. And that's in the future, it will be seen as almost impossible that you don't validate things with your users the same way that today you won't put something into production if it hasn't been tested.
So that's what excites me about research is that if we create the tools that empower and that allow these futures to happen then that's a massive success for all of us.
[00:25:44] Erin: Yeah.
[00:25:45] JH: Yeah, totally. It feels like even just a couple of years from now, like two, three years in the future, the way that some of this stuff's going to be incorporated in like streamlined within teams, it's like, it's just hard to imagine. Right? Like I think we probably don't know all those ways yet. And as we start to see them, it's been really exciting to see what teams are able to come up with and how they incorporate this stuff.
[00:26:00] Jo: Exactly. And just like we said, I mean, I was a web designer 15 years ago, and now we had all these fancy titles. I think that seeing researchers and then UX researchers in quantitative research, isn't quite as even research ops exploding. It just feels like the timing is right for research to evolve and mature and get to the place where it needs to be.
[00:26:19] Erin: Joe, thanks for joining us. This has been great.
[00:26:22] Jo: Thanks for having me here.
[00:26:24] JH: This is great.
