Highlights:

• [01:50.94] Roberta’s taco: when research goes from frustration to satisfaction. 
• [02:45.21] How people’s perspectives influence research decisions.
• [03:49.55] The pros and cons of planning, and how seemingly arbitrary milestones are useful.
• [06:11.90] JH’s Taco: customer segments, moving forward, and cool new opportunities.
• [08:54.76] Work over the course of a year leads to a big product relaunch.
• [11:07.59] Tide pod time: tension between researchers and product people.
• [14:22.78] Strategy gets lost from infighting – can’t we all just get along?
• [16:06.60] Research needs to be high quality and timely to be effective.
• [17:43.49] What operating model works best? Does research get the respect it deserves?  
• [18:53.13] Dynamics of working with designers, engineers and product people.

About our guest

Roberta Dombrowski is the VP of User Research at User Interviews, as well as a career and life coach at Learn Mindfully. She has spent her career leading research teams and enabling researchers in various industries to better understand their customers. In addition to her work as a UX researcher, she is  known for being an excellent cat mom, gardener, and thought leader. Visit her profile to learn more.

Transcript

[00:00:00] Roberta Dombrowski: Everyone's focused on right now the current quarter, the changing market conditions is like huge of what we've been talking about and hiring, being put on hold and not so much shifting to the year ahead yet.

[00:00:19] Erin May: This is Erin May.

[00:00:21] John-Henry Forster: I'm John-Henry Forster, and this is Awkward Silences.

[00:00:34] Erin: Hello, everyone and welcome back to Awkward Silences. Today we're here for our fourth, maybe fifth, something in that range of tacos and tide pods with our guest Roberta. Thanks for coming back. We're excited because it's September, the best month of the year, and I've got a sweater on for the first time in a while, and it's feeling very cozy. Today, we are going to do, instead of three tacos, three tide pods, one taco, one tide pod, and just get slightly deeper into each. As a reminder, tacos are good and tide pods are bad. I think today, Roberta, you have a taco for us.

[00:01:15] Roberta: Yes. Been really enjoying time with the team lately. JH's team product has been driving a lot of vision strategy work for not only like Q4, but 2023, and it's been like a lot of fun. It's actually reminded me a lot of my product roots, but from a research perspective, I haven't seen a team get this far ahead in a while. It's been cool to see even our evolution from last year when we were talking about the vision and it's also become a signal for one, seeing into JH's brain, which is like, whoa. Then two is it gives a signal for research. Where can we like get ahead for really the next few months and start to probe and pick at hypotheses and assumptions and stuff like that. It's been like a nerdy time on the team lately.

[00:02:08] Erin: Well, that's the best time fitting for September back to school.

[00:02:11] Roberta: Yes, that's true.

[00:02:13] Erin May: Are you finding that you're also able to, I'm putting a leading question, but bring research insights into that planning process. Like research insights we already have, things we already know to inform the vision, and how we're thinking about it.

[00:02:26] Roberta: What a great question, Erin. Yes. As like JH has been iterating their thoughts and their questions, I've been going into our research repository and trying to pull up anything and everything related to the topics that they're swirling around to get some sort of signal. I went from, oh my God, I hate this. This is taking so much time to like, oh, I found this, like now I want to go do more research because I think the existing research is a good type of signal about the current state, but you can't like, you're not talking to a customer and asking them, like being able to probe deeper. They're identifying trade-offs with that process too.

[00:03:08] JH: I would agree with all this. I definitely think the macro level includes research in your planning. I think that's a good taco. I think the thing that's cool about it is I think as you were joking about the way I work, been all over the place with different ideas. I think what's helpful when you have the research perspective involved in planning is you can figure out where do we have enough conviction that we actually don't need signal here versus like, where are the real forks in the road that this is an interesting question or decision to go get more signal on.

I feel like that's what we've been circling around is-- this thing we all actually agree on, and we think we have enough evidence to just roll with it. This one, actually, there's a choice here that we don't understand. Let's go a little deeper and we haven't figured that all out yet, but I don't think we would've been approaching it that way without some of your involvement in including others in it too. I think it's helpful to get those perspectives.

[00:03:52] Roberta: Yes, for sure.

[00:03:53] Erin: I think one of the biggest struggles with planning is always all models are imperfect or whatever they say. It's like you have to plan, life's going to change. You're not going to actually execute the plan as it was planned, but it's a useful process anyway. This is particularly true with annual planning. Here's a very long time for a startup, however, it's necessary for all sorts of good reasons. We need to anchor the team to a vision and this story and like, what does this all add up to?

This is all really good stuff, but just curious how we're thinking about some of those trade-offs of like, how deep and big and why do we go in terms of this plan versus, ultimately high conviction, Q1, less conviction Q4 2023. Having, I guess not a capital A, but an agile sort of planning process and how we see that playing out next year.

[00:04:45] JH: I'm somewhere in the middle of all this because I think there's, like, something nice to having some rhythm and cadences to the year. Like, "Oh, it's coming towards the end of the year, so we think a little bit longer term because it's just like a natural thing that happens." Part of me likes that, it's like arbitrary and probably not ideal to only do it that way. I think you want a rolling planning where you always are reassessing what's up in the next horizon and shifting as you go.

I think if you're doing planning once a year, that's probably not a good thing, but it's nice. New year comes around and people make resolutions and stuff and maybe I don't stick with them, but it's a shot in the arm to do some things or I'm not as opposed to some of the arbitrary things is maybe others. The thing I always say to my team is, I forget where I heard this, but like a 100-meter race is very arbitrary, but if we all agree that that's a good way to determine the fastest person, it's meaningful, like who can run that.

It's end of the year and start of the year is arbitrary and quarters are arbitrary, but if we all work off that cadence leaning into it and harnessing how that manifests in the rest of the business, like sales works on quarters and everything else. If you understand it, it's actually useful, because otherwise, you get a little too removed from the practical realities of how the business is being run.

[00:05:50] Erin: I have not been too deep into the planning yet, trying to hit singles for the quarter right now. This current one that we're still in, but what are you most excited about so far to noodle on?

[00:06:04] JH: I think for a long time, we've thought about the way we do things at like a very product level of like, we have our recruit product, we have our research hub product and it's a way of finding target users. It's the way of managing your own existing users for research purposes. I think that's served us well as we've been gaining traction and leveling up the business in different areas. I think now we're at a point though where we can think a little bit more broadly and holistically in the sense of, if we are solving like this participant need for people when it comes to research.

To do good research you need participants, and we help you with that and across all facets, I think we're starting to realize that there's some gaps and some cool opportunities when you think that way that we have maybe had on the back burner or a little bit further off that are starting to come into focus. We can share more at some point. I just feel like I found myself not coming at, like last year to put a more tangible example on it.

I feel like I was writing a lot of the stuff for the teams very much from a product perspective. We're going to do this on recruit and we're going to do this on Hub. This time around, it's actually more of like, as a business, we're solving these problems and we have these gaps we need to do and its actually very little current product offering language because it's a little bit more strategic and understanding of the user needs and where we can add value and that's been cool.

I think it gives us a little bit more-- It's going to give the team to think a little bit more discretion for how in those areas we find the best opportunity to move forward. We got to take it down a couple of levels still, but I think the high-level story is feeling like crispr and easier to communicate even though it's more zoomed out, if that makes any sense.

[00:07:27] Roberta: Yes. I think for me, it's been really interesting to see, like I mentioned, like the maturity shift from last year. This time last year, we were prepping for the offsite, really digging into one of our products more deeply, and this year, I feel like our understanding of our customer segments is definitely different and more nuanced and the conversations that we're having about the vision is really interesting where we're talking about different segments. What are the customer needs behaviors? Are they even that much different from each other? Are they overlapping?

I'm finding myself personally saying it depends a lot more than I ever thought I would, which is like obviously a UX term phrase, a lot of people use, but I never thought I would be saying it this much personally, but it definitely applies. That's been really cool to see. Sometimes I feel like my brain is all jumbled because I've been in this space for a little over a year now. I'm sure, JH, Erin, you're picking up so much signal all the time from whom you speak with. It's been cool to step out and look at the actual data that we have, the insights from interviews and piecing it altogether too.

[00:08:40] Erin: Yes. It's been cool. You were talking about a year ago and the research we were doing, I think that was on research hub. It was like early--

[00:08:46] Roberta: It was. Yes.

[00:08:47] Erin: Early research on research hub and then you did the presentation at the offsite and then that led to more research-on-research hub, which drilling in further. Then we did the competitive on research hub and then we did our positioning work, all leading to a ton of product launches and features we're building big stuff. Then next week, we're going to do a big relaunch go to market moment. That's really like a year in the making. Some of that came out of a planning process and some of it just came out of the evolution of what do we need to do now? What do we need to do now? What do we need to do now?

[00:09:18] Roberta: Yes, it's really exciting. It's really hard for researchers a lot of the time to tell the story of impact, but this one is very clear to see the throughput, which is exciting.

[00:09:32] JH: All right. Quick awkward interruption here. It's fun to talk about user research, but you know what's really fun. It's doing user research, and we want to help you with that.

[00:09:41] Erin: We want to help you so much that we have created a special place. It's called userinterviews.com/awkward for you to get your first three participants free.

[00:09:53] JH: We all know we should be talking to users more, so we've went ahead and removed as many barriers as possible. It's going to be easy. It's going to be quick. You're going to love it. Get out there and check it out.

[00:10:01] Erin: Then when you're done with that, go on over to your favorite podcasting app and leave us a review, please.

[00:10:10] JH: It's weird that it’s also about a year that feels like a very long time to plan on, of like we don't know nine months from now how certain we're going to do this in this quarter, but also it's not that much time. To your point, even in smaller businesses for some meaningful product things to add up and some understanding to come together and to get a few iterations out and stuff, it can sometimes take a year to make a real dent in some bigger problems. It's this weird paradox where it's like sometimes, you have to make a bet of that commitment to move a hard area, but it also feels like a long horizon to be guessing what you're going to be doing that far out.

[00:10:42] Erin: Roberta, I know you're tight in a lot of research communities, are you seeing other researchers getting to the planning processes now ahead of next year? What's everybody up to right now?

[00:10:55] Roberta: It's funny, I actually run a UX Research Leader group, and we'd meet once a month. Nobody has been talking about next year yet. We're pretty far ahead. Everyone's focused on right now, the current quarter. The changing market conditions is huge of what we've been talking about and hiring being put on hold and not so much shifting to the year ahead yet.

[00:11:21] Erin: All right. Well, planning, got to do it. It's fun. I'm excited to do some soon of my own. That brings us to Tide Pods. I think, JH, too, had one this week.

[00:11:34] JH: Yes, I think this is maybe small data points here, but I've just seen a couple things recently on some LinkedIn posts and on Twitter threads and stuff, where it's felt like there's a weird amount of divisiveness happening between maybe product folks and researchers in some orgs, some signs of dysfunction, or not having cracked the ideal collaborating model. The things that came to mind, I saw one person posting about continuous discovery and what do you think about that or whatever. It was just like all of the replies were just trashing it and being like, "Oh, TM is doing garbage research. Sounds great," or being facetious, but that kind of stuff.

Then some longer posts on LinkedIn I've seen that were really like, "If you want research to be bought in your organization, the only answer is to hire a ton more researchers if you can. There's no other solution and stuff," which is maybe true. I don't know. I don't have a horse in the race, but I think where I come at it from, and I think you see this a lot in high functioning healthy product teams, in particular, because I think there's been some of these tension moments that have happened between design and engineering and TMs and so forth is you got to find a way to get into a productive debate intention where everyone brings their superpower to the table, and UX is coming at the problem from a different angle and engineering is coming at from a different angle and product is coming maybe from a little bit more of a business perspective or whatever.

If you actually get that together in a really good way, then you get these great outcomes, and everyone enjoys the collaboration and respects what people are bringing to the table. If you don't, then everyone retreats to their silo. You start throwing things over the fence, and it's not a great way to build things. I just worry that there's a bit of that happening in the research space where product people are well-intentioned that they want to get closer to users, and they're seeing all this stuff about research, and so they're doing their best from a place of good intentions, but maybe not executing it in the best way.

Then researchers are often outnumbered in organizations, so they maybe don't have the bandwidth to jump in and help all those places where they would like to, and so then they're focusing more on the craft and skill to make sure people value what they bring to the table. It just feels off in some words. I can't put my finger on it, but I hope that trend changes or hoping that we can change it a little bit. I don't know. It bums me out when I see it. I want those groups working together. I feel like in some companies, that's not the case right now.

[00:13:39] Roberta: When JH and I were talking about this earlier in the week, I was talking a little bit about hierarchy of needs. In organizations, I feel like naturally, it makes sense when teams get defensive. I've met a lot of researchers who are like, "I don't want products doing anything because they think it gets--" a root question that we have is like, "If they start doing it, what's the value that I add as a researcher?" They might be--

[00:14:05] Erin: That's an insecure place to be coming from, right?

[00:14:07] Roberta: Yes, which is hard, but it's also valid. I can understand teams that might feel that way. I think it's how you build trust in that experience together. It's not me against them. Ideally, in organizations, you are working towards one vision together as a group, and obviously, for theory and all that. I think when JH talks about a healthy tension, that is inherent in the way that we work even internally, and we get to better solutions because of it.

[00:14:41] Erin: Totally. I like what you're saying about having one vision because I think this actually threads this planning conversation in this one, which is that in a high-functioning organization, you have a clear strategy and vision that changes as it needs to or whatever, but you have one, and people across departments and teams and pods and whatever know what it is. That's how you settle debates, right? It's not like, "This is my turf. This is my territory. This is my whatever it is." It's like, "Well, if we're trying to do this and everyone knows that because we talk about it all the time. It's second nature. All we need to do is figure out how to do that, right? That's what we're trying to do." We're not trying to like, "Well, I'm the researcher, and I'm going to do this and use my voice in this way." It's, "How can I use my skills to get us where we agree we're trying." I feel like that basic, that can get lost actually really easily when you go into turf mode of, -

[00:15:38] JH: Totally.

[00:15:38] Erin: -"This is my function, and this is what I do here." It's like, "Well, that's not really the point. That all just exists, so we can get things done, right?"

[00:15:46] Roberta: Yes. If we could erase all ego from the workplace-- [crosstalk]

[00:15:50] Erin: I don't know. Hot take, egos aren't a bad thing. It's okay to have pride in your work and do good work and be proud of your work.

[00:16:02] Roberta: Agreed.

[00:16:02] JH: I think the things may be missing in terms of-- you need two things for research to be most impactful as possible. Like what I say. You know what I mean. To be impactful is you need, obviously, high-quality research. You need to be getting good insights that aren't biased, from the right people, and I think, obviously, researchers are going to be really strong in that regard. That's what they do. It's their specialty, right? Then you also need it to be timely. You need those insights in a spot where it can influence a decision, or it can be incorporated into work that's happening.

It doesn't necessarily mean it has to be super-fast or whatever, but if you don't have it on some calibrated scale, then decisions get made without those insights, which is really unfortunate. I think if you get into the partnership mentality, it's like, "Well, the product people are probably going to be a little bit more over-indexing the timeliness of we're working against this cadence or whatever, and we need some insights here," and the research people are going to over-index a little bit on the quality side, obviously, because it's their craft and what they do.

If you actually put those together, then that's when you get the best of both where you're like, "Okay. We're going to get the research on the timeline we need so they can impact these things. It's going to be better than if we had done it ourselves." Then it's like you can imagine a different world where it really comes together nicely, and everyone still gets to bring what they do best to the table. I don't know. Just optimistic that, hopefully, teams can figure that out.

[00:17:18] Erin: I wonder, too, if some of it-- because research is evolving so much and because we've got the democratization, I'd be curious to read someone who's thought about this in the history where people who do research, product managers, product designers, they've been doing research for a long time. This is not a new phenomenon, right?

You've had that, and then you have researchers, and you have, "We're trying to get a seat at the table and convince organizations we need to be here." That happens, but in terms of an operating model of how different teams do their own research and work with researchers, that's different in every organization. Everyone's figuring that out. I think that probably has a lot to do with some of this us versus them stuff is just people don't know how are we supposed to effectively work together.

[00:18:03] Roberta: We talked about this at the Research Leader group this week actually about work design and structure and the trade-offs between one over the other. It depends. I think it's not clear. I know that there's two people in our group that have similar org design to what we have where the researcher reports into the CEO. I feel like at a history level, we talked about how research has owned its chops now. It's starting to be respected. In a lot of orgs, it's been nested within design. Is it time to break it out of design and so that it's like a peer to design for reporting into product or even pulling it out from that, too? People don't know yet. It's happening live, in real-time.

[00:18:55] Erin: On Twitter and LinkedIn.

[00:18:57] JH: Is it what I think of a lot I forget if it's in the book or one of our blog posts, but Teresa Torres and her continuous discovery stuff, not actually talking about research, but just talking about the dynamics between an engineering lead, a product lead, or design lead, and how you need to have the shared ownership and working toward the same goal, an example that she gives in one of those like I said, a blog post or the book, basically, if your designers working a really important design heavy thing that has a ton of high fidelity stuff and they're moving forward and only they can do it, obviously, they need to go there.

If the team also has some lower fi, simpler UX tweaks that they need to do that they would like to get some signal on or do some light project on and you have a UX-minded engineer who could make really cheap versions or a product person who could do a low fi sketch of it and show it to somebody, if you have that interpersonal trust and understanding that we're all pulling towards the goal, and it's like, "I'm using my design skills on this very hard design thing, and I trust you all with some oversight to do enough to move us forward while I'm busy," that's a really cool model.

It's not that they are designers. It's that they're working within the constraints and that there's no bandwidth for design right now. They understand that they all need to achieve this goal and by these other people pitching in in an unusual way for a momentary thing is helpful, from a team perspective. There's just that trust and connection that just needs to come along. I think it is tough when I think in many organizations research is outnumbered. They don't have the bandwidth to have a researcher in every pod or every squad or whatever. That probably makes it even trickier to navigate.

[00:20:19] Roberta: Because sometimes it makes it easier to pick your battles, so humans. I have that conversation with our team all the time. If we're going to go to market pricing updates, it's like, "Yes, we need research there. The pods, they can figure it out. We'll switch into coach mode, and they can drive it."

[00:20:36] JH: Yes, so what I say is, if you have good stories about how healthy and well-functioning your product and research relationship is, you should also get on social media and share those because I don't see as many of those.

[00:20:44] Erin: Oh, you know that person’s there. They got a thread coming, how to run an organization in twenty two or less

[00:20:48] JH: Yes, the good story. Share those ones too.

[00:20:50] Erin: Yes, totally. Yes, and say what's up to us on Twitter. We want to hear from you guys, User Interviews. Right on. All right. Well, we're at time, over actually. This is the evolution of our format. Let us know what you think somewhere somehow and see you next time.

[00:21:06] JH: Okay. See you next time.

[00:21:10] Erin: Thanks for listening to Awkward Silences, brought to you by User Interviews.

[00:21:15] JH: Steam music by Fragile Gang.

‍