Highlights
[02:01] An insider’s look into how this year’s State of User Research was conducted
[05:19] The rise of organizational buy-in
[06:06] Measuring the different facets of impact
[08:44] Demand as a signal for impact
[10:45] The growing role of research operations
[14:11] Reporting and team structures
[16:07] What democratization means for professional researchers
[18:19] Helping distributed teams limit bias and analyze data
[19:57] Why it’s a great time to be a user researcher
[29:07] Growing momentum in your practice
[30:44] Predictions for the future of user research
About our guest
Roberta Dombrowski is the VP of User Research at User Interviews (and a User Research Yearbook Class of 2022 member—check out her profile here). In her free time, Roberta is an adjunct professor through Boise State University's Organizational Performance and Workplace Learning (OPWL) program and mindfulness teacher.
Resources mentioned in this episode
Transcript
[00:00] Roberta: Personally. I have a belief that a lot of the creativity and experimentation in the industry will come from research ops in ways that we can think about solving a lot of the problems that everyone is going through. And that's also like, I have to catch my bias a lot when I'm doing research. If I identify themes from the surveys or even like talking to researchers, that's something I have to gut check all the time.
[00:29] Erin: This is Erin May.
[00:33] JH: I'm John Henry Forster.
And this is Awkward… Silences.
[00:43] Erin: Hello, everybody. Welcome back to Awkward Silences. We are here today. We've got Roberta Dombroski, VP of UX research at User Interviews, the biggest company in the world, back for the second time. Welcome back, Roberta.
[00:58] Roberta: Thanks for having me again.
[01:00] JH: Here to, yeah, we called the State of User Research SOUR, internalizing an acronym, and I'm just seeing a thing right now for this recording that says Roberta SOUR.
So Roberta’s actually in a good mood, but it's just a funny combo.
[01:11] Erin: We are talking about SOUR today, the state of user research. This is now in its fourth year, which is fun because we get some sort of longitudinal data here in terms of, you know, things we've been asking about for years and have observed sort of anecdotally, but now can quantify somewhat.
So we'll dig into that. And some of the changes we've seen over time, and this is Roberta's first year being part of the State of User Research, both to talk about what we've seen as a researcher, but also, as someone who helped put the report and the survey together. So let's just take a moment, not too long, but let's get meta for a minute and just kind of talk about:
How did we put the State of User Research 2022 together? And how were you involved? What were you excited to do this year versus maybe being on the other side, having read it in previous years?
[02:01] Roberta: You know, this was a very meta experience for me. I have been reading the report for… really since it started.
And so I was really excited to dive in this year. Definitely a huge collaboration with the marketing team, Katryna, who's not on the call, really championed and led the whole effort. And so I worked very closely with them on the plan for the survey putting together what outcomes we were trying to get.
We wanted to definitely measure benchmarks year over year to see trends. But there were also some areas I wanted to explore that I had heard in qualitative interviews from customers too. So we added some new areas this year, which was exciting. And so yeah, we put together the plan, like any research study.
Crafted the survey and it went live and all the work after that came in analysis, and it was a beast. Say it a lot. It was all hands on deck, but it was a really great experience. And seeing it live now is just it's the first time, Erin, you've had like really marketing visuals, like Holly was on it for the first time, too, our graphic designer, and it just looks amazing.
And when I saw it, it's just like a proud mom moment, a proud research moment whenever you see findings from a study, so I was really excited.
[03:20] Erin: I'm just, I'm wishing maybe we can spin up a userinterviews.com/sour. I think we should probably do that, but for now you can find it, Google it, go to our website, we promoted everywhere.
Yeah. Holly did an awesome job. Thanks for mentioning both Holly and Katryna, who definitely definitely did a ton of heavy lifting here. So one of the things you mentioned was you're excited to not just benchmark, baseline, compare across previous years, but also ask some new stuff. So what were some of the things you're excited to add in?
[03:50] Roberta: Oh my gosh. So a personal area that I've been wanting to explore is a research impact. There was a blog post that Lizzy, one of our content team members, made a few months ago and I've been obsessing about it. And so I added a section in the survey on impact. How do you measure the impact of your team as a research?
I also wanted to dive deeper into tooling. We do the tooling map every year. I've been obsessed with the map and the visuals every year. Internally, we're thinking about integration partners. So I changed some of the sections of how we asked the questions and thought about groupings of different types of tools too, so that we could use that data.
And then also that the way that we were talking about it, aligned with how researchers were speaking about it from conversations and calls with them. I also have been thinking a lot. I have this in the back of my head, maturity is coming up as a team of research practices. And I hear a lot about org design like team structures from researchers.
So I included a new section around reporting. What team, what function do you report into inside of your organizations? Is it product research? What is it to kind of get like themes there as well? So there were quite a few like new sections this year. But luckily it didn't feel all that different from previous years, which was nice.
[05:19] JH: I have a very interesting thing I'm curious to get your take on. There was the question or the section around, you know, the impact of research. And one thing that was surprising to me when I went through this for the first time, was that around a third of researchers report, basically aren’t tracking the impact of the research that they did.
Right. But then we've also seen year over year. Is that there's more buy-in and support for research at an organization level than ever before. Right? Like we see that improving year over year. I can't figure out if these are contradictory or complementary, like, is it that just because everyone's bought in, that research is valuable and it's really hard to quantify, like the ROI or something, people just aren't doing it.
And there's just kind of like they've made the leap of faith and like that's happening or is there like an actual gap? Yeah. No, we should be closing the loop and like saying that this research was impactful and, and this is the type of decisions or things that had informed. I'm curious how you think about it.
[06:06] Roberta: It's funny because I'm coming off of, I do a monthly research leader group, and this week we talked about the impact of research because I've been obsessing about it. I think there is something there where part of the impact is, is it measurable? There's different facets of impact that I'm starting to just have as a practitioner, there is the actual you're influencing decisions around research, which is more quantifiable.
You can count the studies. You can count the, oh, we learned this, we changed this in a design, but then there's also the qualitative part. That's like, oh, we changed the culture. Like we have more buy-in for research. It's really hard to measure that it's things that as a leader, you see happening over time and it takes longer too.
It takes like two years, at least like it's any time in growing a team. And then there's like the external cultural impact of research, like in society, which is also really hard to measure. And so I think part of it is like, impact is really hard to measure. Depending upon the facet factor of impact. And then also like when you have more buy-in for research, like we have a great team here.
I don't need to justify the number of studies that I'm running, like in counting the number of customers. Obviously I want the, as a data point to tell the story, and some of the researchers that we heard from, cause this was like, the impact question was actually free response. So there was a lot of coding data, but a lot of people talked about like, I don't measure it or I do, when I have a research repository, people will engage with it. And so that's impactful. That's a signal that I can use.
And so I think people are also trying to figure out what the signal is. Because research is still new in many ways. So the signals will vary based on your context, always.
[08:02] JH: Yeah. The product answered everything. That's like the top part needs to interview PMs. They just say, it depends on everything you think will change next year. Do you think it'll go up or down in terms of the number of people reporting?
[08:14] Roberta: I hope so. I mean, for me as a practitioner, I hope that like every single researcher I speak to every week, my perspective is changing.
And I think the more that we talk about it as practitioners, I think other people will be experimenting. And so it's likely we'll see a change in it next year. I hope we do.
[08:33] JH: You think you'll see more measurements next year?
[08:35] Roberta: I hope so. It might not be a quantitative measure. It might be more qualitative observations of signals in a structured way.
[08:44] Erin: I think the demand signal is really important. Like that's something you see in the data is that. People are demanding more research. And of course, researchers are going to do more research, that's their job is to do research, but people who do research are doing more research. People who do research, and I'm talking about powders, we call them PWDRs, but anyway, they're doing research and they're doing it because they value it.
Right. It's not like they have time for it. If it's not valuable, as opposed to researchers where it's like, this is my job. This is what I do, and you're seeing more demand for it. Right. And so that to me is a very clear signal of impact because why would you demand something that takes time if it isn't useful to you, or if you don't perceive it to be useful for you?
So that to me is one of the strongest signals to your point. We're still trying to figure out as research teams, like how do we measure impact in a structured way? But that is something that's happening. That I think might account for some of that seeming paradox of like, we don't know how to measure it yet, but it feels like there's more buy-in. It feels like there's more, buy-in people are doing more research. They want more of it.
[09:47] JH: Yeah. Yeah. This is a silly way of maybe thinking about it. But the first thing that comes to mind is like, if you think about driving and you're like, what's the value of paying attention while you drive or having your eyes open? It's like, you don't get in an accident and you don't quantify, like, if I had crashed into that thing, you know, I would have had to replace my car in this.
Research has kind of that, right? Cause it's like we were going to ship this crappy thing. We did research and we shipped something that's better. And like quantifying that delta every time I think it's just really challenging.
And so like, once you get people to believe that like, oh, it's good to drive with your eyes open and pay attention. It's like, oh yeah, it's good to do a lot of research as we're developing this stuff. And that seems to be kinda what you were saying. Erin is like, you can measure it more on the demand side, maybe than the impact on every signal occasion.
[10:21] Erin: R and D. It's not D. It's R and D.
[10:24] Roberta: Yeah. And I think, like you're saying demand is a signal, I think the growing headcount is also a signal that we're having an impact. Like companies wouldn't be approving, hiring more researchers if they weren't being impactful in some way.
And then also we wouldn't see a rise in non-researchers doing research too, if they weren't seeing value in it. Right.
[10:45] Erin: Well, let's talk about that. Let's talk about, there's a rise in non-researchers doing research there's a rise in headcount of researchers. All that kind of touches on a couple of related themes, one democratization and two research ops.
So let's dig into those, you know, research operations as a field was kind of like that, is that a thing? When I started here four years ago, when we first started the state of user research report. And now it very much is a thing. It's not a thing every organization has someone doing in a dedicated capacity, but it is a discipline, whether it's a dedicated role or not.
So what jumped out in the report as far as research ops and democratization to you, Roberta?
[11:28] Roberta: Yeah, I think for me, and even speaking to research ops practitioners too, is that it's very rare still. Like it's definitely growing, research ops is becoming more prevalent. It's very rare to have specific roles for research ops.
We find that they're in larger organizations and then in smaller sized organizations, it’s always various obviously, but in smaller size organizations, research ops ends up being a component of a researcher's job or a person who does a researcher’s job. Hey, you need to do recruitment and talk to a customer. Like it's just part of the way that you're doing your work and getting value.
And then it's really wonderful as companies start to scale, you want to streamline all that work that's getting done. And that's when like the specialization, the specialist role can come in, which is really exciting to see. I personally have a belief that a lot of the creativity and experimentation in the industry will come from research ops. And ways that we can think about solving a lot of the problems that everyone is going through.
And that's also like I have to catch my bias a lot when I'm doing research. If I identify themes from the surveys or even like talking to researchers, that's something I have to gut check all the
time.
[12:46] JH: I'm thinking back on the conversation we had with Kate Towsey from Atlassian a while back where I think the rule of thumb she had around research ops was like, once you hit like eight researchers, ops starts to be valuable. That was her measuring stick.
And to your point about, it's not that common yet. I was like, well, it's probably not that many organizations that have eight full-time researchers, but I wonder if something that maybe people haven't figured out the calculus on is like how you account for people who are doing research.
They're not dedicated researchers. So if you have four dedicated researchers, but you have 12 PMs and designers doing research, are you in a spot where you actually need ops at that point? Maybe the dust hasn't settled on that thinking yet.
[13:20] Roberta: Oh, totally. And even when we were analyzing the data, I was like, oh, I wish I asked this in a different way when we were trying to figure out, like, org structure.
Because it is very nuanced. So trying to count that you're, it's really trying to think of the whole system inside of a company and who's doing it and who's running what part of research, the practice, are they doing? And it's hard to really capture that in a few questions or even a conversation with someone.
[13:50] JH: And the adjacent part here is around the whole thing of people who do research powders and the democratization or whatever you want to call it. It's kind of linked into this, right with, you know, we have a lot of demand. And so there's more researchers, the research ops becomes more prevalent, but you also have people who are not researchers doing research.
And so you have these different models happening. What were some of the takeaways or trends in that area?
[14:11] Roberta: It's interesting. So much of my brain is exploding right now. I think one thing that came to mind was team structures and reporting structures too, with all of these different groups of people who are doing research.
One thing that we found is that people who do research are really distributed across teams, like it could be product managers, designers. When you get to a point of having research operations, it's typically that person might be reporting into research or design. It really varies by like org structure.
And so there's like hybrid models that are actually coming to be where a lot of teams will have like centralized research teams or centralized design teams or product teams, wherever researchers reporting into. But then researchers will also be embedded as well into the context that they're working.
And that has everything to do with different facets of how research work is being done when like tooling. How are you quantifying studies, measuring the impact, all of that. And that's also why I wanted to dig into the nuance as much as I could of the team structure, because it influences how the work is being done so much.
[15:24] Erin: I just thought it would be interesting to triangulate our SOUR data with our internal app data and see the job titles and the sizes of these different teams. And who's actually on the platform and using it right, because when you respond to a survey, you know what you know about what's happening in your organization, but you don't know what you don't know about.
[15:47] Roberta: Next year! That was the stretch goal that I had for SOUR this year. So we'll see if we can get to it, but yeah.
[15:53] Erin: Democratization. Research ops. What else jumps out to you when you were looking through either comparing this year's trends versus previous or something that you were kind of interested in on the new side?
[16:07] Roberta: So before we did this SOUR report too, I talked a little bit about bias because I'm a researcher researching research. I actually created a document with a massive list of assumptions that I had so that I could compare and contrast what I'm noticing in my work. And then what's actually happening. Like, is this accurate?
Is this an observation? And one of the gut feels that I had was we're seeing this rise of people who are doing research. It's likely going to be like evaluative research rather than discovery type research. And that was really cool to get validated on that. That was a theme, like a pattern that did emerge.
And the survey this year is that with the rise of democratization, when researchers are doing research, it's typically generative and then other teams, people who are doing research will do more evaluative design, usability study type work, which is really great. I think there's a lot of fear around democratization for researchers.
I think about it all the time. And so with that, what that means is that if we do open up, which we've already been doing, open up research to not researchers, it means that we can focus on strategic work. Which is like what everyone as a researcher loves doing. Like, that's why we study our craft so much.
So that was really cool to see a pattern that emerged.
[17:34] JH: I feel like maybe just because I do this for most things, I can kind of talk myself into different beliefs and points of view. I could almost see that going the other way to some degree. Right. Cause like when you're in the stage and doing usability stuff, that probably means the designer or the PM has like some sort of concept or something that they brought to life that's like relatively firmed up and they're like kind of attached to it and they probably have some developers like ready for work and it feels like they could go out and then really like butcher the research.
Cause they're just trying to hear what they want to hear. Whereas, like if you plugged into a professional at that stage, hopefully that would go better. Whereas that's probably still true in the generative, like discoveries I do, but maybe less. So like if the, if the PM is out there, just like hearing pain points and starting to like pattern match, I don't know, but I kind of go back and
forth.
[18:19] Erin: That's why I know, Roberta, in our recent panel conversation at the advancing research event, we talked about, how can centralized researchers really help these distributed teams? And one of the things that came up was helping teams to say eliminate bias because that's impossible, but to check their bias, acknowledge their bias. And that's where it feels like that could really come in. If, you know, evaluative research is something that people who do research are doing the most stuff that they're finding the most useful.
The most sort of like fits into their workflow, like, okay, do it, but write down your doc of assumptions or whatever it is to make the research worthwhile.
[18:59] Roberta: Accurate. And it's something that I've been thinking about for our team internally, too, cause we're exploring more evaluative work. And how do you have things like analysis anchors or like quantifiable measures when you're doing evaluative that all ties into the quality, the rigor, putting guard rails on things. I think about this very deeply. Yeah.
[19:24] Erin: Well, because that feels like a risk on the generative side. If you have, you know, people not being researchers, people are doing research, doing generative where on the one hand, anyone can find a user with something to say and ask them good questions and learn something, but then taking all of that qualitative data and turning it into signals.
Okay. Okay. What should we do with this? Really useful to have someone trained on analyzing qualitative data to help with that part of it? Right. Good time or bad time to be a UX researcher.
[19:57] Roberta: Good time. I always tell my team, especially after the Sara report came out, I was like, well, we have job security. We are in demand right now.
And it's really cool to see. Like qualitatively, you know, we get the updates, we're seeing articles throughout the year about job postings, but, and even salaries too. It was really cool to see that people are really happy, a really strong signal of people being happy with their roles this year. We also saw strong signal as salary increases, which there's been a lot going on with a great rising nation this year.
But yeah, it's really cool to see. The trends that have happened over the past year and how it's surfacing into the research space itself.
[20:41] Erin: It looks like over half of folks with five years experience or less are making a hundred, 250,000 salary. Two years ago, just under 25% were making the same.
So, you know, definitely reflective of the macro trends, but also seeing that within UX research.
[21:00] JH: I dunno if we have the data on the company side stuff there, but I wonder if we can pull that up too, with, as the big companies that grow their teams, if sometimes there's a little bit of a higher salary band.
I wonder if they're also as they invest further and further and research helping lift that up.
[21:31] Roberta: Yeah, and we did some slices. I don't have it in front of me right now, but in the fuller report, we talk a little bit about it and I believe we're going to do some follow-up reports as well into different data points that we collected.
But we looked at role level and salaries. So, differences between director and management. And then I see is, and we also looked at company size since we have that too.
[21:37] JH: All right. Quick awkward interruption. It's fun to talk about user research, but you know, what's really fun is doing user research and we want to help you with that.
We want to help you so much that we have created a special place. It's called user interviews.com/awkward for you to get your first three participants free.
We all know we should be talking to users more. So we went ahead and removed as many barriers as possible. It's going to be easy. It's going to be quick.
You're going to love it. So get over there and check it out.
[22:06] Erin: And then when you're done with that, go on over to your favorite podcasting app and leave us a review, please.
[22:14] JH: I guess the question I have to weave some of these things together is… it's a good time to be a user researcher. You know, companies are bought and salaries are going up, people see the value in it, but then you ask to have all these non-researchers doing research.
Maybe just like from the conversations that you have with that community, Roberta, and maybe more anecdotally, like, is that like a good thing? People are liking it. Because there's more awareness across departments. That research is valuable and people want to roll up their sleeves. Is it still a little bit weird of like, there's kind of two camps and a little bit of judginess of like, you're not doing that very well.
What's your like, pulse or vibe on that?
[22:46] Roberta: Yeah, it's interesting. I'm just coming off of a conversation with John Cuttler yesterday. We actually dug into the different lenses at which research is happening inside of our organizations.
I gotta preface, I'm really weird. I've worn every hat. I've been a designer. I've been a product manager and decided to really specialize in research. So I understand all the perspectives and how research is different in each of those contexts. Generally, what I've noticed is that from speaking to researchers, there's fear around things like the quality of research. When we talked about bias earlier, when you start to open the door of, oh, more people can do research, more can go wrong.
And I think also there's fear tied to what does that mean for me? I've spent so long perfecting my craft. Many times people go to master's programs to get a PhD and it’s like, we don't learn this overnight. And so. Part of it's fear around, like, what does that mean when we have other people doing research? And that's where I think like, it's happening, we know it's happening.
And so as practitioners as a researcher, what does that mean for me? Like we have the agency to figure that out, which is really awesome. Like that's the energizing part. As a leader, I can help figure that out. I can create structures and training to put guard rails, and really think about the nuances of how research is being done on our team in different contexts.
And so that's one thing that, you know, that's my own bias too, in my perspective, but that's something that I want to encourage researchers to think about is like, how do we view it from an opportunity standpoint, in an enabling standpoint? Because it should be done. It is a craft and it should be done well, but where is also the flexibility in how it's being done for other contexts too?
[24:45] Erin: What do you worry about when you worry about more people doing research? You're talking about kind of, you know, a lot of folks have advanced degrees and a ton of training and it's not something you just come out knowing how, how to do research. Right? It's a skill. So what are the risks?
[25:03] Roberta: Yeah. I think for me, what I think about is seeing a signal.
What you think is a signal and then confirmation bias. It's not actually a large signal. You're just noticing this to confirm something that you were checking, like you had a gut instinct for us. So that's very scary. I think also the analysis part that we've talked about before too, is depending upon how you're doing research, adapting, if there's different ways to analyze things.
I love down and dirty affinity mapping. I prefer it, actually. I'm also someone who's very flexible in my own methods because I come from a different background. So I love that, but I do get scared about, are we quantifying the qualitative insights in the best way? Is this representative of wider signals?
Those are probably my personal biggest fears for things. And then I also like—Roberta controlling research. I want to be involved in everything. I love research. That's what I want to do. I want to be with the PMs like nitty gritty and ring out the evaluative studies. I love doing that just as much as I love doing the generative work.
It's like a reality check for me too. Like you can't do it all. And I think that's part of it for researchers too, when you democratize, it's like, you actually can't do it all. You can't use research to inform every single decision in the business, if it's just you doing it, but how do we enable other people to use it as well?
And then also maybe we'll hire more researchers. So yeah, those are some of my personal things that I struggle with.
[25:47] JH: Yeah. Yeah. I think the fear in the other direction is like, I'll put on my, a powder hat of the chronic side. I think a lot of product people don't necessarily want to own, like specializations that are not their wheelhouse. I'm thinking about myself, right?
Like I've done design work in a pinch because we were short-staffed or whatever. Right. I find that stuff interesting. And I like learning it, but no, I'm not a true designer. I've got examples. I'm happy to let like a real designer come in and do a much better job.
Right. And same thing with research was like, I think this is a problem that benefits from research and I'm happy to be scrappy and do it myself and get some signal that would probably be better than doing nothing. But, you know, having somebody who's a real practitioner come in and do it would certainly be.
But I think the fear is that when you, as many in there's going to be like, somehow that's going to slow things down where it's like me scrappily doing the design poorly. I think I can get it done in two weeks and we can move forward. Then somebody comes in and does design full-time and tells me it's gonna take six weeks.
We got to meet in the middle here somewhere. And I think that's probably a little bit what people are working through on the research side, too, of like, I want you to do this. I'm sure you're going to do it better than me, but I also can't wait two months. So we got to figure out a way to, to get on the same page. Like the turnaround and the approach and all that sort of stuff.
[27:54] Roberta: Yeah, for sure. It's funny. You mentioned that. Cause I was talking to Paolo earlier this week. Who's one of our senior PMs. We, on a recurring basis, I mean like jam on discovery approaches to discovery. And he actually said that. And usually when I'm interacting with a person who does research, like a designer or PM, they'll come to me with something that they're interested in discovering, and then I'll go through like, oh, you could do it this way, this way, this way, this way options to try and mitigate that.
Like if I had more time, I could do it this way, but you can do it this way really quickly and cut some corners. And that's where I feel really grateful for my design and product background. And also, I think I'm not sure how other researchers outside of like user interviews, if they're getting down into like here's option ABC.
But I could see that being really helpful as they're starting to build out how they're working with other people and like democratization is helping. Yeah.
[28:55] Erin: Like that, go find your pragmatic researcher who will help you cut the right corners to get just enough signal. But sometimes it's like a need.
Right. And that's how you work together in real life.
[29:07] JH: Do you think about a contractor or something? Right. Like, I want some help, like doing my interior design and my living room or whatever. And you reach out to people in your, like, I have a budget of like, I don't know, $2,000. And people are coming back to you and they're like, I'll do it, but it's gonna be $15,000.
And we're gonna knock down a wall. You're like, that's just too far away from what I am grounded on. And like, maybe we can do a project together. That's close to my budget. We stretch up to 2,500 whatever. And then next time we'd use something I'm willing to stretch more because I trust you. And I saw like, what a good job, but like, you can't just come in.
And I'm like wildly at differences of spectrums. And I feel like sometimes that's what people in these different roles can get themselves into a little bit.
[29:39] Roberta: I am a huge fan of, especially when partnerships are new and democratizing research is new and different, like growing into a new team. I'm all about the down and dirty, quick wins.
Do we do useability tests? Let's get feedback, let's share it with the team. And then once you have that trust, you can start to expand into the more like, hey, we could do this really cool study with all these methods and triangulate and all these super interesting ways, but we're not gonna be able to get to that if we can't help the team inform a decision quickly sometimes. Yeah.
[30:13] JH: There was a great phrase I saw on Twitter the other day from somebody. I'm not gonna be able to attribute it cause I forget where it came from. But, it was describing what you're doing in product, in product development.
Just being like, you're trying to just develop like a momentum engine. Like you just want the team to have momentum on solving problems. And I think that goes with how research comes in and how design works. And it's not just like, you need to feel like you're making progress against whatever it is you're trying to deliver.
And like momentum is relative. Like how fast you need to be going or what that looks like is going to be situational. But, I thought that was just a really nice way of expressing, like what you're trying to get the team to do together is just like being in rhythm and building momentum. Cool.
[30:44] Erin: So first year, doing user research, what are you…. or are you resting and recharging, but what are you thinking about next year for, or what are you curious to see? What might change any predictions?
[31:03] Roberta: My predictions. So I definitely see just the number of researchers growing and the impact that we're having on teams continuing to grow. I really envision the people who do research.
So. Product managers, product marketers, designers. I see that rising. I don't see that stopping anytime soon. I see salaries continuing to grow too. I think it's because I get obsessed. Like that's what makes me a researcher. I get obsessed with these random things and I just want to keep tearing them apart, but like, I really want to dig into how we're measuring impact.
I really want to try and figure out the signals and start to triangulate the signals more of org structure, team models and stuff like that. It's the greater context, like how a team is structured and who they report into that influences happiness. That someone has within their role and how they perform in their job, if they're able to do a specific type of work or that yeah,
work structure was the biggest reason for unhappiness.
And so, I'm really curious to see. Our org structure is going to shift as democratization shifts who researchers are reporting into on teams. Is that going to change over time? Right now, I report centrally to the CEO. Are we going to see more of that? As research becomes more prevalent because the company has realized the impact. It's not just product decisions or designs that they can influence, but it's sent to her company. I'm a little biased, but that is a trend I would like to see and I think will happen over time.
[32:52] Erin: What will grow the most? Between now and next year research shops, researchers, or people who do research as a group, people who do research, I think.
[33:03] Roberta: Yeah. We shall see. Yeah, I think so.
[33:11] JH: I think so too, but then I'm also like, I feel like just from the people I know, like in product and design rules, across from companies, like they're already doing it. So we might be like a little sash room. I've heard of being tapped out on the side. I don't know. We'll see.
[33:24] Roberta: I do think that the number of researchers is gonna continue to grow too. I think it's all gonna grow, but I think those two.
[33:33] Erin: Yeah, it could be interesting next year to look at just the amount of time spent doing research among these different categories and the percent of research happening in an Oregon who is doing it. And by doing it, we really have to break that into parts, right?
The planning of talking to users, that analysis. Right, because there's so much there. So unpack in terms of how has the research been getting done? How has the research been used? Who's doing it? Who's spending their time on it? Because to JH's point, probably a lot of people. I think we put it in terms of like, are you spending 10% of your time on research that makes you a person who does research?
It's not the majority of your time, but it's, you know, a consistent piece of what you do. But there's obviously a lot of range between 10 and a hundred. And you know, maybe what will happen with some of these hybrid models again, predicting, I don't know, but I'm curious to see if researchers are spending more of their time helping and assisting people who do research or people who do research saying, I'm not the expert here.
I don't want to spend more than 10% of my time on this, but I very much want to spend 10% of my time on this. So I think that'll be interesting to see how that breaks.
[34:46] Roberta: I think we'll also see a rise in research coach roles. The more companies start to democratize because we're even shifting to that. I'm finding my percentage of time is shifting to doing more coaching.
And I know it's certain types of companies. Some enterprise teams are hiring specifically research coaches to come in, which is exciting to see.
[35:08] JH: So yeah, a couple of things I'm curious about are one, if we'll see organizations try to more explicitly define ratios of like researchers to engineers or other functions, like you see that with like design and stuff sometimes, right?
Like for every X engineer, we want a designer and like we're going to budget for that and stuff. I wonder if that will come from research. I'm very curious if we'll see movement between roles, like people who do research, who really like it and have some aptitude for it, move into full-time research or full-time researchers who are embedded and really like the strategy piece, jump over to product or vice versa.
I'm curious if we'll be able to pick anything up there. I'm super curious. The general tool, like prevalence of zoom, just being so common because of the pandemic and ever moving remote. Everyone knows how to use it. And we already pay for it. Like if that will maintain or we'll see more specialization creep out.
And the last one, this is kind of adjacent, but I think this year, I think somebody is going to crack the repository stuff in a way that really clicks in is like, you know, not like goes viral, but like catches on. Like, I think you just saw that with design systems, like there was a lot of movement and tooling around it and like thought, and then at some point it kind of tipped over.
Okay. Everyone is kind of doing design systems this way now in terms of how they manage components with developers, everything, I think we're close to that point on the research
outside.
[36:19] Erin: Yeah. Well, let's take that further. Cause when I, back to four years ago, I remember JH one of the first articles you shared with me was—remind me his name at, yeah, exactly. The nuggets.
And I know there's still that atomic research, the air table, and you know, some of that's definitely still happening, but of course there's blow back to that and all sorts of different views on, well, you can't, decontextualize an insight and it has to be part of a larger sort of study and in context or whatever it is.
So, yeah, I'm curious if anyone has. That's JH, are you going to crack the code?
[36:58] Roberta: I smiled when you said that, because we're knee deep in repository work. And I was like, ah, it's working. He's thinking about it.
[37:07] JH: No, I'm not. I don't think I'm going to be going to do it, but I just think we're close. It just feels like there's been moments that have stuck and resonated with people like that.
Kind of like an atomic unit insight thing. There's a lot of tools right now for managing inside. Everyone's kind of capturing their research artifacts digitally or in a way that is storable. And so that's kind of exploded and people are wrestling with it. So it just feels like there's a lot of like the factors kind of there that someone's going to like to put those ingredients together in a way that I think really resonates with folks.
But we'll see.
[37:37] Erin: I agree. I mean, there's definitely pain there. Opportunity. I don't know what it looks like. I'm thinking of Grain, which I was using earlier today, is just such a delight to use for just pulling clips, which is very different than pulling a high volume of insights across a high volume of projects.
Right. Then there's AI. Like how does that fit in? I mean, all we talked about in my last job was, AI this and Mr. That and everything, but it does feel like we're hitting a new wave in web three. We haven't even talked about web three. So let's say another episode, but does AI have a place in when you think about just evaluating this sheer volume of qualitative insights and how humans could possibly ever, at scale, do that. I don't know what the solution is either, but I’m curious.
[38:25] Roberta: It makes me think about when we were analyzing the survey data, we had a lot of open response questions and I just put up my hoodie and I went on to coding. So I do think there's a lot of opportunity. I did it in a few days. Cause I was like let's park.
[38:48] JH: I was listening to a neurologist on a podcast recently. And he was talking about how people can focus and stuff. And it was, this is just a super random thing, but he did mention that pulling up a hoodie honestly helps. Because it blocks your field of vision.
It's like, there's a physiological response of, by narrowing your field of vision, you focus more, so good work.
[39:06] Roberta: You should see my setup. When I do analysis, I got everything. Yeah.
[39:11] Erin: I think we've got viral potential here. We're going, gonna send us your sick hoodie analysis pics, put them on Instagram, hashtag research.
[39:23] Roberta: It'll be good.
[39:25] JH: It's state of the union strong.
[39:33] Erin: We sit strong somewhere until next year or until next time we'll get you back on sometime sooner. You're listening if you made it this far, and there's a topic you want to hear Roberta talk about, find somehow on Twitter or elsewhere and we'll put it in the mix.
[39:52] Roberta: Yeah. Yeah. That's good. Awesome. Thanks.
[40:00] Erin: Thanks for listening to Awkward Silences brought to you by User Interviews.
[40:05] JH: Theme music by fragile gang.
%20(1).png)
