Highlights

[2:48] A few reasons your organization may not believe user research is important (yet). 

[8:41] How to get through to relationship managers who don't want to give you access to their clients for research sessions.

[13:52] Be the research change maker in your organization.

[17:04] Generate buy-in over time by keeping stakeholders updated on research progress.

[20:04] Start with evaluative, tactical research to get some quick wins and show the impact of user research.

[26:46] How to know you're having an impact on the research culture at your organization.

[30:54] Ultimately, it’s your relationship building skills that will help you get buy-in for research projects.

About our guest

Meg Pullis Roebling is the Head of CX Research, Data & Analytics Solutions at BNY Mellon. There, she has built and democratized the CX Strategy teams’ research and insights framework across the enterprise. Before BNY Mellon, Meg worked on design and research teams at AIG, American Express, AKQA, and Digitas. 

Transcript

Meg: [00:00:00] I think it's important to not come in and say, you guys are all you're doing this all wrong and here's how you do it because you'll make Some enemies, and it's really important to build relationships with all stakeholders and really understand, you know, what's their experience  

Erin: [00:00:39] Hello everybody. And welcome back to awkward silences. Today, we're here with Meg Pullis Roebling, the head of CX insights, data and analytics solutions at BNY Mellon. Today, we're going to talk about selling and research when your org isn't fully buying in. Meg thanks so much for joining us today. 

Meg: [00:01:00] Thanks for having me. Great to be here.

Erin: [00:01:03] Got JH here too. 

JH: [00:01:05] Yeah. I mean, maybe just to clarify a question I had right off the top, like when you say the org isn't buying in, does that mean they have like research resources, but they're just not using them or they're just hesitant to do research at all or what is not buying in mean and just to start maybe there.

Meg: [00:01:18] I think it, it really means that it's not fully mature in terms of how they're building products. Right. So they're probably not doing that user centered design process that's so important. They may be doing parts of it, or some of it, or maybe some different teams are doing it, but others aren't, it's just not baked into the culture.

So, a lot of education is often involved 

Erin: [00:01:49] Gotcha. Yeah. And I think that's a great point when we talk about getting buy-in . We're talking about, if you can make it part of the culture, it's not such an uphill battle. It's not so hard every time. So it's probably part of what we want to do is try to influence the culture, to see the value of research. 

Meg: [00:02:06] Yeah. And and then once it is, once people, once the company or organization is more evolved and you know, it is baked into the culture then I think that there is going to be more buy-in for the research part of it, because they're really understanding the value of it.

But for myriad reasons, it isn't there yet. And so people don't just, They just don't immediately understand the value of research. So, I would say that's not buying in part of it.

JH: [00:02:38] Right, right. What are some of those reasons? Like, so obviously it's part of the culture that they do things a different way, but like, what are some of the underlying beliefs or whatever that are kind of causing things to be that way? 

Meg: [00:02:48] Yeah there's definitely a few reasons. Some people might confuse it with other research that's possibly going on in the organization. They might think that it's market research, or that it sounds and smells like market research, and there's a marketing team who's already doing that kind of stuff.

Or they might confuse it with some competitive research that's being done by some strategist over here or in this pocket over there. Or very often it can be confused with just general relationship conversations that are happening with clients. So at a highly structured large organization, there's typically relationship managers who are responsible for the overall relationship of that client.

And they have regular conversations with them. And, they're collecting feedback in many ways from the clients, but it's not feedback that's actionable or structured in a way that's accessible to product teams. It's just kind of, on a different level, the things that they're talking about.

Erin: [00:03:54] Right. So are they, is the idea that like research is already happening and therefore we don't need more or research is already happening and we're not even sure if that research is valuable. So why would we do more or, like w 

Meg: [00:04:07] No, I think it's more that they think that they're already, they already know. They're like, oh, we already know what's happening. We've got market research, we've got client conversations. Or they think that talking to users means doing a demo of the product. So it was oh, hi, here's our, all of our news.

Here's our new thing that we built, and this is how you use it. Instead of actually listening, right? Act like setting up a structured conversation in a way that's going to elicit unbiased feedback you know, allowing those awkward silences.

JH: [00:04:42] Yeah. Yeah. So how does our user research or find themselves in this position? Right. So they're hired and they're in a company that is kind of not fully bought into research, but they hired a researcher. So. There's something there that they have good intentions, but they need some help getting over the hump.

Like how does that situation come to be?

Meg: [00:04:59] Yeah. I think that for sure, there's always going to be a bunch of people, hopefully a critical mass of people in the organization who understand that there is a better way of building products, right? That doesn't involve waterfall, that isn't engineering driven, that is, user centered process.

And typically those are the ones who say, we're going to start doing things a different way and you know, start championing a project here and there, or a hire here and there. Right.  

I think it very often comes from UX designers who need and want more information about their users and about how their product is being used in order to do design. So, I would say that it very often starts from within the design team or from a knowledgeable, experienced product manager. 

Erin: [00:05:52] So what are so, okay. so. There are pockets in the organization that see the need for research. There are other pockets that say, yeah, research is good, but we already have that. What are some of the other things that might come up that could be obstacles for someone, in a well intention or part of the org, they're going to come up against some issues, getting their research done, getting buy-in. 

Meg: [00:06:15] Yeah. I would say there's sort of two, like bigger categories for people objecting. One of them is. Is the time and expense required around research. They may be allergic to the word, even research because, maybe in the past they've hired an outside company to come in and do a pocket of research.

And they, these companies come in and they spend two months, they don't interview users and that creates a big deck and it feels very disconnected from. The people working on the product day to day. And I think that these people have become kind of distrustful of that process for whatever reason.

They don't feel it's super valuable. And also because it's just, it takes a long time. It can take a couple of months to do an in-depth qualitative study. So, we can talk about ways to get over those obstacles. But I'll just mention another area that, that comes up a lot when I'm trying to convince stakeholders to do a study, is that, oh, Hey, we've tried this before with this other team or this other person.

They talked to our clients and these are, this is an objection that typically comes from relationship managers. And it is that, we've done this before. We gave you access to our clients. You show them all these great things, and then that thing didn't get built and we look silly. So, that's something that needs to be managed as well.

JH: [00:07:55] Yeah, I feel like we've kind of talked around this topic with previous guests and I feel like there's a couple things that have come to mind. One is the importance of almost doing research in building trust and fluency with your stakeholders. So like taking some time to investigate internally, like what's going on and what are some of the blockers and having that context.

So you know how to build relationships and overcome it. And then the other kind of perspective we've heard with some frequency. Just get some of these people in the room and show them the magic of, like, a one-on-one interview and like show them some pulled clips or have them take notes on a call and like, realize, we don't know everything that we think we do.

Is it, should people be looking to take one of those two things? Do you deploy both of those as part of your toolkit to make changes here? Or is it something else entirely in your experience?

Meg: [00:08:37] Yeah, I would agree with both of those tactics, right? Just like really listening to the relationship manager and saying, Hey, like tell me what happened before and why did that leave a bad taste in your mouth? And, here's how we can do things differently. Right. We can set expectations.

Part of this process is getting them more accustomed to showing work that's not perfect and finished. They think they may think that's a bad thing and that it makes us look bad. But I think in the end you're there usually clients especially are usually kind of stoked to be involved in the design process. So it's just about the positioning. Hey, we're going to show you designs in progress? And because you're, you're a special client and we value your feedback so much, we want, we want to let you peek under the hood and help us. And I've gotten so much great feedback from clients.

Hey, thanks for involving me in this. And this is going to be cool. And so. Tell stakeholders these stories as well, but you do really have to build trust and show them right. Getting them in the room, inviting them to the meetings. And I know that there's, you know, a lot of thought, that's like, you really want unbiased feedback from clients, you shouldn't invite the relationship manager to the meeting. Because they won't. Feel good giving you the unvarnished truth. Like this thing is bad or I don't need it. But I always invite the relationship managers because especially at first, right. Because they need to see how this meeting is being run.

They need to gain trust in you. And I have found that typically I invite them to the meetings. They come to a few and then they're like, oh, This is good. Right.  And they just say, okay, you know, send me the recording later. 

Erin: [00:10:27] Yeah, they get it. So before we dive too deep into solution territory, I want to make sure we understand. So we talked a little bit about some of the problems. Stakeholders, blockers, people you need to convince, to get buy-in. Might have you, they've had a previous negative experience.

They think they already have enough research. It's going to take too long. It's not a good ROI. It's not worth the effort. Like all these sorts of things to put it in context, right? Like, so these problems don't exist in a vacuum. They exist within organizations.

So what are some of the different, like sort of, maturity levels, different stages a company might be in, are there sort of like company personas or types that you might find yourself in that you might be able to pattern, match and say, Okay. like this is the situation I'm in and therefore, these might be some better ways to handle getting buy-in based on the larger context that I find myself in.

Meg: [00:11:29] I guess, in terms of like, thinking about maturity, like the spectrum of maturity and enthusiasm, I think. I think to your point it's good to have an understanding of where you fall on the spectrum of that maturity and willingness to embrace research and it can help you set your expectations about what you can get done and actually what your job is. Because I think, you shouldn't go into a research role at an organization that is digitally immature thinking that you can just, you know, get a running start on various research projects and get going on stuff.

You should probably be aware that you are going to have obstacles and that a lot of your job is going to be around building the processes and educating stakeholders about the value and not just educating but showing the right brick piece by piece. When that is, this is the way to go.

Erin: [00:12:27] Right. And so if you're a, you know, whether you think of it as a spectrum or a pyramid, or like, whatever, visual, right? If you know where you are, you can kind of say, you know, at my tenure here, I'm going to try to move that forward or up the pyramid, but I'm not going to just like. I need a roof to self-actualize overnight, right?

Like we're gonna make incremental progress. And that feels to me pretty important in terms of just managing your own expectations about what, what a person can actually accomplish. 

Meg: [00:12:56] Yeah. And to know what excites you and what you're good at. Right. Like I like building things. I like helping organizations. And so, this is a good role for me. I enjoy sort of seeing the slow and steady progress against, I guess, I guess I like a challenge.

JH: [00:13:14] I'm going to, I'm going to guess a little bit here, but it feels like as a researcher, there's gotta be some tension between. Like doing things the right way. Like I want to come in and show people what great research looks like and really do it rigorous in a way that is going to have an impact.

And there's going to be meaningful and gathering, good insights versus I need to get some wins and I need to just find a way to insert myself. And even if I'm cutting some corners and maybe not doing things the way I'd like to do it, ideally I'm building some trust with a stakeholder and helping them in some incremental way.

Is that like a real trade off that like a person's going to have to wrestle with. And where do, like, how should they think about that type of dynamic?

Meg: [00:13:52] Yeah, I think it's important to not come in and say, you guys are all you're doing this all wrong and here's how you do it because you'll make Some enemies, and it's really important to build relationships with all stakeholders and really understand, what's their experience with this process.

Like I said about the, like talking to those relationship managers who are like, we've done this before and we got burned, we ended up looking stupid and so really hearing them and, or a product manager who is like, look, I have the vision for this product already. Don't come in and tell me that what I'm doing is that I need to hear from other people.

Cause I already know. So you need to really listen to these people and say, okay, what is this vision? And you're bringing it obviously from some place of expertise. So educate me, right, fill me in, and then like, let's work together to figure out what questions need answering. And we'll set about getting those answers together.

So I think it's very much about relationship building. I would say is one way to think about it, but, If you're feeling just resistance all around. I think that in part, in the process of building those relationships you'll find some allies, right? You and I think it's just really important to focus on those people first.

If you've got an ally or two, one relationship manager that gets it, or one product manager that gets it and wants to work with you that's where you start. And that's where you start to get those wins. 

 Erin: [00:16:09] As we're talking about this. I'm sort of imagining like the new kid in town, like I'm, I'm like a new in a company and I've got big ideas for research. And I, I don't want to blow it by telling everyone they're messing everything up. Right. But a lot of people have maybe been in a position for a while.

Right. Or like you've been there for years or a while. And you. Or kind of realizing slowly but surely or, oh, this place isn't quite as bought into this stuff, as I thought they were, maybe you've even tried. I don't know. You've tried to build some relationships or you've tried to show the impact of your work.

Like coming up a little short. So, I don't know any tips for pivoting or like keeping the momentum going, like one thing I imagined can happen is not every research project leads to a silver bullet insight. That's going to transform the business, like how do you show the impact of research over time?

If every project isn't like a total game changer? 

Meg: [00:17:04] Yeah. I always try to keep people involved and keep stakeholders involved in the research by dribbling out little tidbits as we do them. So, I invite, as I mentioned, I invite stakeholders to interviews, but then during the analysis process, maybe to send out a little like, Ooh, this is a thing that we're hearing, or here's a quote, right.

Just kind of to keep them engaged. That's one trick that I've done. And another thing that is like, when something kind of lands with a little bit of a thud might be, somebody says, well, like we knew that already. Right. And that's always a bummer when you're hearing that because, and I, what I try to do or try to answer when I hear that is, part of this work is not necessarily discovering, Amazing new ideas, but it's building empathy, right?

So maybe, that this is a problem area and you're working on it, but just to sort of really hear in real words, or see somebody struggling with an interface or a process, then, you begin to build empathy that you then maybe you didn't have before.

Erin: [00:18:24] right there. Isn't like you didn't know. And now, it's like, well, that's great. You know, Like Jackie, that you already knew that, but like, I don't, I made up a name. But like now you know, more, 

JH: [00:18:33] The context. 

Erin: [00:18:34] right. Yeah.

Meg: [00:18:35] Exactly. So, those are ways and then just, gosh, getting that research to be accessible to stakeholders is for sure a challenge. Yeah. These folks are, whether they're engineers or product managers or relationship managers, right. They've got their own fish to fry.

They don't, they don't want to read your 20 page deck. Right. So how, like what, in what ways can you deliver it to them? That will be interesting and keep them excited. I mean, I always do a readout and invite everybody and try to make it as interactive as possible. But you know, inevitably people can't come to the readout or they just want the, like the executive summary and it, I won't say it's not hard, but we try to deliver a few interesting snippets and like during the research, I would say.

And those real moments like those video clips and those quotes, I think are the best way to kind of get to be.

JH: [00:19:29] For sure. For sure. That makes sense in a situation like this, is it any easier to make progress on things like the generative side versus the evaluative side? Or is it so contextual? It's hard to say, but I guess I'm just trying to think through my head of like, is it easier to be like, Hey, I'll help you with your next quarter planning.

We'll do some like high level discovery stuff and you can take it or leave it and weave it in. Or is it easier to be like, Hey, this thing you're already working on it. Why don't we, why don't we show that prototype to a few people and see if maybe, the copy or there's something about it, it's a little hard to understand, and maybe you could tighten it up and you can imagine complexity on either of those as well.

Any thoughts there in terms of which side tends to go better or is it just.

Meg: [00:20:04] Yeah, I would say that. Right. There's you know, as you're alluding to there's a couple of different altitudes of research, right? There's the tactical research. Yeah. Is that something that you're looking at and it's, maybe it's in motion, others, there's a design for it. And you want just feedback on some interactions and maybe a little bit of why there, but, and then there's the more strategic research discovery work or personal work.

Can really help build a more foundational product strategy and both are super important, but these foundational, strategic work is more time consuming and therefore, expensive. So I would say that to build an appetite for that you need, and you need to show wins. Definitely start with the tactical research.

See evaluate. Type of research. And one of our previous places that I worked at when we were trying to get buy-in for doing research, we recommended it. Doing like kind of a benchmark, an evaluative study of their registration process, because we knew that registration and login was, had several pain points.

So we just said, all right, like let's just go through your process here with a few users, see what their problem areas are, get some reactions. And then we would do the same with a competitor site and the results were, I think, pretty eye opening for the stakeholders to relate to. So you really see users like actually struggling with their interface.

So, that was a great way for that. And they were like, well, we need to do this more. So, yeah, for sure. I would recommend starting with more evaluative, tactical type research to just get those early wins.

Erin: [00:21:59] What about folks in different kinds of roles, different kinds of like, maybe you're a person. Research, maybe you do research full time. Maybe you're mixed methods. Maybe you specialize, maybe you're very senior, maybe you're very junior. In these kinds of different scenarios. What do you think about trying to get buy-in from where you are? 

Meg: [00:22:20] So I think that the best way to approach it might be to have somebody who's a more senior researcher or a manager, kick it off, right. Say, okay, we're going to do this research. Because I think that in cases where people are resistant to it, sometimes it's, you need to ask for forgiveness and not permission.

Right?

Erin: [00:22:41] Right. 

Meg: [00:22:42] So we want to have somebody who's more senior involved so that they can give cover to anything like that type of approach. And then and then, more junior people. Couldn't really help with the actual implementation and execution and come up with ways, creative ways to maybe get the feedback where you don't have to go through a regular channel, for example, with recruiting, doing something more gorilla.

So I think that there's definitely a role for everybody. 

Erin: [00:23:12] You need to find some influence, whether that's with your boss who has influence, what do you do when your boss doesn't have influence? 

Meg: [00:23:20] Oh, if your boss doesn't have influence? Wow. I think again, it's about finding those allies, right? Finding the champions, people who get it and who are willing to say, yeah let's do that workshop. Let's do that. Users study let's do that. Survey with people from mechanical Turk or just willing to take some risks, what they perceive as risks to to get some information.

Erin: [00:23:44] Let's say I'm a young researcher or maybe someone transitioning into research, or I dunno, maybe I I'm entirety of startup life and I found like a well-paying fortune 50 company with great benefits and that's really appealing, whatever the scenario I found a company and I get the sense, maybe research isn't a priority.

Do I take the job? 

Meg: [00:24:10] I mean, I would say, yeah, because just speaking from my own perspective, obviously you can really have so much impact on an organization like that and they really need people who get it. I guess you just need to be aware of, you just have to have your expectations in line that a lot of the work is going to be around the relationship building and education part and showing people what good looks like.

These tools and products and services. These large organizations know that they impact the lives of so many people. So the work that you're doing is greatly appreciated by anybody who touches those digital products. So, I mean, I think it's just as important, if not more important to do this kind of work then, then a small startup that can move quickly and yeah.

Build things from a digital first perspective. Yeah, I don't know. I mean, those things are, those are important too, but just, it's just a different focus.

JH: [00:25:06] Yeah, no, that's a great point.

Erin: [00:25:09] Yeah, it's an interesting perspective because I think it's, I'm reminded of, have a good friend who's like a real scientist, like, she's got a PhD, she's does like microbiology, she's going to study cystic fibrosis for the rest of her life. And like, in that kind of career, if you find, if you learn one thing like that, that your whole career is worth it.

Right. And not exactly what you're talking about, but you're saying like, Yeah. It might be frustrating. You can't like, everyone's like, high-fiving you and your amazing research all the time. But like the stakes are so high and like making a couple of key findings can go a long way in that kind of organization. 

Meg: [00:25:48] Totally. And they're often really high impact. You're often building tools for experts and helping them in their job means making decisions based on complex data. So how can you make their lives easier so that they can focus on more strategic work or more important work.

Right. And, just help create tools that help them do their job better so that they can, yeah, they can, they didn't move things forward in a way that's helpful for everybody.

Erin: [00:26:21] Yep. 

JH: [00:26:21] to follow on the point you said about having the right expectations, like what is a fair expectation for somebody joining this type of situation in terms of like, when they might be able to see some tangible progress? Cause like, I think, the larger, the org, sometimes the slower it is to do some of the change management stuff and win over hearts and minds or whatever you wanna call.

Meg: [00:26:39] Yeah.

JH: [00:26:39] probably not like a quarter, but like, is it like the first year, I can make, expect to make some headway or like, how should people kind of orient themselves on that piece?

Meg: [00:26:46] Yeah. That's a good question. And other signals to look for going back a little bit to Erin's question is that you want to make sure, even if it's immature, you want to make sure that people are, the people in charge are at least committing to, we need to do this the right way.

Right? You need to be, you need to hear some important voices. 

Erin: [00:27:05] Yeah. 

Meg: [00:27:07] Either the CEO says client experience is important or the head of your division, it was like, oh, this is a person who recognizes the importance of it and, or, or your boss, but you need to be, hearing those key phrases about, we understand the importance of this and we want to that's why we're hiring you.

Right. That's why we're looking for somebody like you. Before, I wouldn't say don't do it at an organization where they're not even saying the right stuff. So that would be key. But in terms of signals to look for like how you're doing, is this going well, I would say are, have you gotten a couple of good studies out there?

Is it info? Is it being baked into product decisions? Are people talking about your research? Do you hear them? They're, are they mentioning it in emails? Are they mentioning it in meetings? So I would say that those are some indicators that you're starting to have some impact. 

Erin: [00:28:03] Yeah, Now let's say you've been building relationships, finding some wins, like kind of handling everything the right way in terms of moving up that pyramid of buy-in then what do you do? How do you push your advantage? To take advantage of some of that momentum you might be seeing. 

Meg: [00:28:25] So pushing the advantages. I mean, that's a topic I've heard on your guys' podcasts, a bunch of times, right? Measuring impact. How do you measure impact? And demonstrate that the work you're doing is valuable so that you can get additional investment to grow your team and get more resources for research.

And by resource resources, I mean, the tools that you need access to participants in that type of thing. And then of course, resources means more researchers who can take on more projects. The goal would be to just sensibly, go up, grow the practice, the research practice and at a place like a large enterprise, numbers, the bottom line is really everything.

So you it's, and it's hard. It's very hard to measure in hard numbers. The impact that our research has, but do it anyway. Um, in a previous role. We were able to demonstrate that by doing a usability study during every sprint actually saved us time. So we looked at, oh, how long it took the design team to execute a number of stories in a sprint.

Then we started doing user testing for every single sprint and we would do these like little micro studies on something really bite-sized. Oh, oh, should we use a multi-select box versus a dropdown? Or should we show percent sign versus. Present end dollar sign, like little questions like that, that we just were kind of causing friction between the research, between sorry, between the design team and the product team and the development team.

And we would do these quick little studies. They would be in the field for a day. We would get the data back. And everybody would be like, oh, well there's our answer. And we could just move on. So we found out that it actually sped things up by about 30%, 25 to 30% at least from the design sprint side, like the design side.

And that's real savings. Right. And of course this translates into development time saved as well because you're getting more finished designs earlier, right. And less rework and all that, which is less tangible. That number, that 25 to 30% quicker is a real number that I could then go to senior leadership with and say, see, this investment makes sense.

And that was actually really helpful in getting a bigger budget.

JH: [00:30:53] Yeah. 

Does there tend to be some sort of senior champion that like, kind of the catalyst for some of this stuff? Or is it like, I don't know, back to the past experience for myself, it was not involving user research, but it was a change management initiative at a large company from a pricing perspective.

And there were a lot of people who didn't want to change pricing because everything they did was based off of her current pricing. And we did have a very senior person who was just like, look, this is the vision forward. Like we're doing this. And it gave us a lot of air cover when we were trying to convince people.

And I'm just curious if there's any sort of that dynamic.

Meg: [00:31:25] Yeah, for sure. It's extremely helpful. Right. If there's somebody at the top who is championing a vision, that includes this process and who gets it innately.  but I don't think it's always the case, especially in organizations where. Things change pretty quickly. You might have somebody in charge who has the vision who's on your side.

And then somebody gets put in that position who thinks they have the things, they understand the user centered design process, They don't, or they, they think it's something else. And you know that then things get trickier. And then again, you have to kind of, show, don't tell.

Relationships work that are, that are in positions of power, but maybe not at the top or

JH: [00:32:08] Yeah, so you gotta do the same work kind of regardless, but somebody at the top helping you can be like a positive tailwind or something, but it's not going to change what you need to do from a relationship building standpoint. And all the other stuff you've mentioned is that kind of fair.

Meg: [00:32:19] Yeah. A hundred percent, 

Erin: [00:32:21] Meg closing advice. What do you know if you had some advice to give somebody that we haven't talked about yet? When it comes to the fun and rewarding and difficult and. 

Meg: [00:32:32] Yeah.

Erin: [00:32:34] work of getting buy in. 

Meg: [00:32:36] Yeah. I would say that it's an awesome challenge to work in a place where there is resistance or there just isn't awareness, but it's also super rewarding because you can make such a big difference. Yeah. I've had so many experiences where people are really just thrilled, right.

At the outcome of a big study. And they're like, this is amazing. How come, we haven't been doing this all along. Right. And when you hear those words you know that you're doing a good job and it's super gratifying. So, I would say If you can make a difference with your work and you're believing your passion then by all means get involved with a big company that needs help because they do need help.