Highlights

[2:04]The key problems with B2B recruiting.

[8:22] Make your sessions as short as possible, 30-45 minutes is best.

[12:53] How to make users feel connected to research, even if you can’t offer an incentive.

[16:53] How to work with internal stakeholders to get better response rates from participants.

[23:08] How Matt built a study about new Braze users with a lookalike panel of people who had never used Braze. 

[33:09] Steering clear of over contacting participants, even with a small pool. 

[38:09] What Matt's thinking about going into 2022.

🤔 Customer Success Teams: Are they a roadblock or a valuable resource for UX research?

About our guest

Matthew Morrison is a Senior UX Researcher at Braze. Before Braze, he worked on the research teams at Etsy and WillowTree. He’s excited by the challenge of B2B recruitment and is looking forward to continuing to improve his practice.

Transcript

[00:00:00] Matthew: one good thing is a lot of our customers are willing to provide useful feedback. They're in this product day in, day out and they really just want to share their frustrations and also sometimes things that they really like. Hello everybody. And welcome back to Awkward Silences today. We're here with Matthew Morrison, a senior UX researcher at Braze. Today we're going to talk about B2B recruiting and the kind of background that is, I know Matthew, you mentioned when you were trying to do B2B recruiting you couldn't find a lot of great resources out there.

[00:00:49] Erin: And so he came up with some of your own and we're going to hopefully help some folks out here. So thanks so much for jumping in to talk about B2B recruiting.

[00:00:58] Matthew: Yeah, sure. Happy to be here.

[00:01:01] Erin: Got Jay tier two.

[00:01:03] JH: Yeah, this is going to be a fun one since part of what we offer is B2B recruiting at user interviews. So I spend a lot of time thinking about this problem of how we enable it for other researchers. So I'm hoping maybe I can steal some ideas to pull into our own product from this one.

[00:01:15] Matthew: Definitely.

[00:01:16] Erin: yeah. And a full disclosure, a fun one for me. Because I used to work at Braze. So it's a love, love that little Braze love.

[00:01:22] JH: TrailBrazer? Do people say that like you're an

[00:01:24] Erin: sure. Why not? Definitely. Awesome. Well, Matthew, thanks again for joining. Let's just jump right into it. B2B recruiting. So, you know, you mentioned when you were trying to do some B2B recruiting, you were looking for resources, but didn't find a ton of information out there, which to me says, maybe there were some challenges with B2B recruiting.

So let's jump into that. What are some of the unique challenges when you're trying to recruit participants for B2B research?

[00:01:50] Matthew: Yeah. So just as a little backstory, I started at Braze last year. So I've been there almost a year, but I'm used to doing B2C recruiting. But I have done a little B2B in the past, but yeah, lots of interesting challenges behind B2B. I would say the first one. like access and because you're B2B, your customers have customer success managers, account managers.

So there's a little bit of gatekeeping that happens there. It's not necessary to go through them to gain access to customers to talk to. But you know, for certain types of customers, More sensitive ones, bigger clients. It's a good idea. And it's a good idea to keep them in the loop. So that adds a lot of overhead.

And it kind of varies. Some customer success managers want to be on the call, want to be kept in the loop or just want to be CC'd on an email that you send others don't care at all. So that gatekeeping part is probably one of the more interesting challenges.

I think people don't often think about it. And beyond that, I would say the biggest one is you have a limited pool. So Braze has over a thousand customers. And by that, I mean companies worldwide. And that sounds like a lot, but when you really break it down, like you have at Braze, we have designers and product managers also conducting their own research. Over time, this number dwindles. It isn't so big. We often are, you know, trying to, we talk to customers over and over the sometimes same customers because of that. So, it also can be difficult if you want to recruit a specific type of customer. So for example, our platform has a lot of features and quite a few different types of users in it.

It's mostly marketers, but we do have developers. So as an example, if you want to talk to developers who integrate Braze with their company's tech stack, for example. So like when a company buys Braze, they'll have their developers come in and integrate it with the other types of technology they have.

The developer number is even smaller and they're even less engaged because they're not active in the platform day to day. So they're even harder to reach and harder to get interested in talking to us and just lack knowledge about the platform itself. So those are the key ones, and then there's a few others as well.

[00:04:10] JH: Yeah. And on the access piece, like the gatekeeping kind of dynamic, I'm just curious to kind of clarify that a little bit. Is it just like, Hey, it's a new thing and I want to be aware of who's talking to my clients, is it, I don't want you showing features we might not build cause like, what if they love it?

And then, you know, we don't move forward or, or like, relationship stuff. I'm like, Hey, they're in the middle of a renewal process. Like it's not a great time to like to throw more stuff at them. Like, what are some of the objections or why that gatekeeping thing comes up in the first place?

[00:04:37] Matthew: Yeah, I would definitely say the first and the last one you said. So the first one being like, they just want to be in the loop of, Hey, you're talking to my customer. I just want to know what you're talking about maybe and like, just so that they're aware that this happens so they could document it somewhere.

You know, like at, for conversations, like documenting things in Salesforce or a CRM of some sort, and then also the latter, like just a relationship thing. Like I have gotten, it's been really important, like where I have gotten I picked out a certain customer. I wanted to talk to an account manager and they said, actually, it's going through a renewal process right now.

Now it's not a good time, which is good to know, but it does add a little bit of that overhead.

[00:05:19] Erin: And I imagine the kind of key insight here, right, is this is going to vary from company to company. And from, you know, from account manager, customer success, manager, sales might get involved. Anyone who has a relationship with the customer that's going to vary and I'm curious. You know, our you know, and B2B in general kind of that ownership of the customer relationship can be very interesting.

And so, is that something that's maybe evolved, you know, at Braze in your time there? Or is it something that you mentioned you've got product managers, designers. Lots of people want to be talking to customers, which is great, where we come from. You know, that's a wonderful thing talking to customers, but you also don't want to bombard them or over contact them or create any kind of negative experience for customers.

And, yeah, I'm just curious how you've found ways to, to navigate that.

[00:06:11] Matthew: Yeah. So I would definitely say that's something that's evolving at Braze and we're currently working on because yeah, the more people doing research, the more touch points it has. And if it's not connected in some way, like there's no kind of a centralized way. Like how many times we've contacted this person for what?

Then it gets a little tricky. We're working on it. And like I said our customer success managers account managers, they've been nothing but wonderful. And they're always like, yeah, talk to this customer, unless there's like a specific reason why they just want to know. And I would say there's ways that we don't always have to contact them.

We have different ways that we recruit through Braze. If it's usually like a specific customer, you want to talk to them. Then you should usually loop in, you know, the customer success manager or their primary contact at Braze. If it's something more general that your target customer is, you know, you want to talk about Braze at large and you don't really have anything specific.

It doesn't matter what type of stuff. You can just kind of send out like a blanket email or what our platform does is send, you know, an in-browser message or an in-app message. And we can actually do that through our platform and we can recruit customers that way and they can see if they see a message like that they can sign up on their own without.

The customer success manager needs to approve. At first, I actually am doing something like this literally today and recruited some customers that way. And obviously I'll follow up and let them know that they've signed up. Let the CSMs customer success managers know they signed up, but that's one way they kind of get that gatekeeping doesn't have to be such a barrier.

[00:07:52] Erin: Got it. So, you know, to just to summarize what we've talked about so far, there's kind of in B2B, uh everything's relative, but relatively speaking, you've got a smaller pool and it can be harder to access the pool that you do have. So those are definitely some notable challenges when you're just trying to find relevant people to talk to. What are some of the other things that can make it, you know, more challenging to recruit B2B.

[00:08:16] Matthew: Sure. So, you know, one is, you know, and this goes for everyone really, but their time is limited. It's valuable. So we've kind of, and I can talk a little bit about the solutions that we've worked around this, but, you know, we have more luck recruiting our people, our customers, if we limit a study to say 30 minutes versus an hour. Problem is sometimes you need more than 30 minutes.

You need an hour to actually do the full, like whatever your project is. And so that is an issue. And, you know, you kind of have to meet people where they are and they're working. I don't, do we say nine to five anymore? Nine to six? But more than that, like they're working long hours, so you have to meet them where they are.

So I usually set my availability a little bit earlier in the morning and a little bit later in the evening. So, and again, like just today, I got a bunch of people signing up for something like a 6:00 PM call or maybe a 9:00 AM call.

To talk to me and that's great. Like, I feel like that's really important because they don't feel obviously comfortable taking time during their Workday to do something like this. So that's one thing we've noted. and I would say another thing and I think a lot of people might be thinking if they're listening to this is like, what about incentives?

Can you offer people incentives? So currently at Braze we have not offered incentives or gifts to people as an overall program. And there's a couple of reasons behind that. So like some of our customers, they won't allow their employees like our users to accept gifts or payments from us or from vendors in general.

So I think some people are hesitant about that. In projects where I have offered an incentive, which I'll get to in a moment, like I've had people tell me they can't take the incentive or they don't want that, but they're happy to help, which has always, I mean, that's really nice, but you know, you don't want to scare people away by offering an incentive either.

But we're realizing the need. Like it's really hard to recruit and as many people as we are recruiting, we're starting to roll out a more formalized incentive program. I can talk more about that shortly, but like a little pro tip was your incentive calculator. Super helpful. On your website.

Helped figure out how much and all those things. So,

[00:10:29] Erin: Got it. So time is money. So that's kind of right? yeah people don't have time and like, how do you compensate them for the time that they do have where, you know, there can be challenges around internally. Like what is our incentive structure, particularly for B2B, where I think it's pretty standard to compensate on the B2C side.

But on the B2B side, it can be a little bit more interesting. And I think the time compromise is really interesting. We were actually talking to someone at NNg the other day, the Nielsen Norman group. You know, it was saying. But if you, as remoteness becomes more popular and it opens all these opportunities to do research with people all over the world, that's great.

But it also can mean, okay, now I, you know, what am I going to do time zone wise, to to folks and kind of making sure, like, as you're adding flexibility for your participants, that you're also not kind of burning the candle on both ends yourself to accommodated, you know, finding a healthy balance there as well.

[00:11:26] Matthew: Yeah, definitely. And so when I say like meeting them where they are, we do have customers all around the world. And I, you know, I'm, my calendar is not open 24 7. If it's, if I'm dealing with, like, if I'm trying to get international customers in a specific region, I will be mindful of that. Like open up my calendar to fit with their times as much as I can.

But you know, for like something like this, where it's, I'm accepting like global customers, I try to just put like a little bit of, you know, little earlier, a little bit later to even help out, like, you know, I'm on the east coast to help out with San Francisco folks in that office out there to get, you know, so they can join in maybe at a six o'clock meeting.

But I generally don't go too late, unless it's a really important project that I'm trying to get to a specific group of people. 

[00:12:16] JH: Yeah, yeah. To go back to the time and money dynamic a little bit. It. Yeah, it's kind of, it's kind of counterintuitive, right? Cause it's like, usually when somebody doesn't have time and you need to talk to them, you offer more money and then, you know, they perk up and that doesn't really seem like an option here.

Is there anything you found in terms of like how you, when you would do the initial outreach and how you kind of explain what you need to do and why it'd be valuable to talk to them that like is a good hook or like pulls people in a way that like, you know, maybe the monetary value can do in like the B2C setting.

Have you found other ways to pitch what you're doing or. You know, pique their interest, so to speak such that they like to engage and respond to you.

[00:12:53] Matthew: Yeah. You know, I can't speak in absolutes. I have, we haven't done any AB testing on our language, which is something we were just talking about today. That's something we can do and we should do to see what works better. But we, I mean, there are things that I've tried some along the lines of like your feedback is helpful in improving the platform.

You know, you'll see your feedback, like your feedback is taken to heart and we actually. If it's something that's a very specific product thing, we will follow up with certain customers about it to show them how, like basically their feedback did make a difference.

So it, you know, it's, I feel like I'm speaking a little bit in platitudes but it's, just trying to get them to feel connected to this and making it better since a lot of the people using our platform are in. You know, all day long, like, you know, you want to make the experience better. We want to make the experience better.

You want to make it better because it'll just make your day better. So things like that, and I wouldn't discount incentives. I'm not against monetary incentives. I was saying, you know, like some customers won't allow their employees to accept gifts, but, you know, I think there's more to it than that. And incentives do definitely help.

To some degree, especially on like we're on time crunches.

[00:14:12] Erin: you also mentioned. Time is valuable. It can be easier for folks to find a half hour than to find an hour, a couple of questions there. One is just sort of a, in the pandemic setting and I guess you've been embraced kind of unfortunately the pandemic the whole time.

[00:14:29] Matthew: Yup.

[00:14:29] Erin: But does that make it a little bit easier for folks to find a half hour in the middle of the day being remote or not?

And then I guess the other question I had is you know, between the half hour of the hour, do you kind of. Start out asking for an hour and you're finding that it's hard to find people, then you'll adjust to a half hour or have you proactively updated your timeframe to just kind of know, like an hour is going to be tough.

I'm going to try to design my studies to be more than a half hour. I can get what I need out of them in a shorter time period. 

[00:14:58] Matthew: So to your first question, I, since I started Braze remotely and everything's been remote since then, I can't, I don't really have a good comparison. I imagine it's a little bit easier to recruit people than it was in the past, but I, yeah, I

[00:15:13] Erin: baseline is.

[00:15:14] Matthew: sure. My basically, yeah. So, and then before that, when I was at a different company recruiting I was, it was B2C, so.

We're able to get people all the time. So, I will say into the second point 

so I think it was a little bit of a learning curve for me. So I wanted to fit a lot in, I started with an hour for one study, one of my, the first studies at Braze and was having a lot of trouble recruiting and learn from, you know, just people who been at Braze for a while that they found that the same thing that you'll get people if you make the study, not quite an hour, 30 minutes, maybe 45 an hour just seems overwhelming to people and like, especially in a busy day. So I've since then started proactively trying to design studies that I can try to keep within 30 minutes. And if not then 45 minutes, 45 minutes I've found is I seem to be able to get away with it, still experimenting with it.

But 30 minutes. Definitely have proactively done, designed to study to kind of fit in that time. And if I need to do more research later then so be it. 

[00:16:22] JH: one of the other things that comes to mind is, you know, when you're trying to get people to talk to you and spend time with you, you have incentives, but there's also just like, you know, all of the different techniques and approaches you can employ in terms of how you reach out. So I'm curious if you found anything that works well there.

So like, you know, you're keeping the account manager or the customer success person in the loop. Do you ask them to kind of do like a warm intro? Do you just reach out yourself from your own email? You know, will you send like reminders, like you, you know, ask somebody on Monday, if they'll talk to you and you haven't heard from on Wednesday, will you follow up or does that feel like too pestering?

Like any kind of practical things you do on that side that have helped you get a better response rate?

[00:16:58] Matthew: Yeah. Definitely I, if a customer success manager, for example, is willing to. Reach out for me and make that warm intro. It's super helpful. And what I've done is provided a template for them. They're very busy and they get a lot of requests and you want to take the burden off them as much as possible.

So I'll provide like, Hey, would you mind reaching out? And here's a template just filling your name and the customer's name and CC me on it. That has worked much better than if I'm just writing it and CCing them. Although if I, if that is what I need to do, I often say, the customer success manager said you would be a great candidate for this.

I'm reaching out to them, CCed on here and see if you'd be interested in a study, but that warm intro definitely does help. And providing a template ahead of time for them to save customer success managers time is super important. So, That's one way and yes, the follow-up emails. It takes a lot of overhead when you go this way, you know, you it's a whole, you know, re ops as a whole field.

Now, like having a coordinator, for example, would be wonderful because, you know, keeping track of how long it has been since I emailed that person, you gotta really be on top of that. And I do that as well. Like if I haven't heard I'll follow up. And I take, I don't put the burden on, you know, a CSM to do that at that point.

[00:18:23] Erin: Got it. Great. So, we've talked about some of the challenges and some of the workarounds. Let's dig more into the topic. So I know you've been able to find a lot of ways to make B2B recruiting easier, including using user interviews. So we'll get to that, but yeah. Talk to us about how you've made B2B recruiting more successful throughout your time.

[00:19:26] Matthew: Yeah. So some of these I touched on already, but I'll just go into a little more detail. So one good thing is a lot of our customers are willing to provide useful feedback. And I think, like I said, they're in this product day in, day out and they really just want to share their frustrations and also sometimes things that they really like.

In that sense, we do have those kinds of really engaged customers. And speaking of our engaged customers, we actually have a community called bonfire. It's a slack community of Braze customers. And we, you know, someone on the Braze team runs this community and helps foster the community there, but it's pretty much like a forum for customers to talk to each other and learn from each other.

But also it's, these people are super engaged and it's become, in some ways a de facto, I wouldn't call it a panel, but we do use that sometimes to recruit like, Hey, is anyone here working on, I don't know this particular type of campaign and would be willing to provide feedback on an idea that we're thinking about.

For improving campaigns, email campaigns, like just sign up you know, provide a Calendly link, sign up here. That's been super successful. So, that's one. Another one I've found is like, we have a lot of customer satisfaction surveys running in our platform for different types of features that we're rolling out to make sure, you know, over time.

We're improving the platform and we keep track of those and customers, not optional, but customers who provide open-ended feedback in those surveys. I've targeted them in the past. Like, oh, they've said something either, you know, about a problem or something that you wanted to talk about and they just wanted to get feedback.

And I feel like if they might be more willing to give that feedback, maybe they'd be willing to chat with me more. So reaching out to them directly has been something I've done. And that's one way of getting around this recruitment issue. I already talked about meeting customers where they are like.

Their hours are different from yours. And sometimes, or even if they aren't, sometimes they don't want to take time in their day, but they're willing to do it at like five 30, you know, which isn't so bad. So those are some of the key ways. And then finally the big one that you know, when you're really having trouble is recruiting users who are similar, but not your actual customers.

So, yeah, this is the example that you alluded to Erin, like user interviews. And so, I can go into detail about that project if you want and tell you how that went, if that's okay with you.

[00:21:57] Erin: To hear you talk about user interviews.

[00:22:00] Matthew: Yeah.

[00:22:00] Erin: No, but we would love to hear about your process because I think, you know, that really brings a kind of recruiting to life in terms of how you were able to do that. And I think. You know, we'll get into this more too, but some of the pros and cons of talking to, you know, your like closest customers who are using your product all the time versus, you know, maybe new customers who are customers, but told, knew super well versus, you know, maybe like prospects who are interested and know something about you, but versus.

Right to your point, like people who look like they could be your customers, they match the general persona you're looking for, but aren't customers and all of those have different pros and cons. So we'll get into that. But one of the pros of talking to people who are not your customers is it gets around some of those challenges you mentioned, right?

Where there are more of them. There are more people who look like your customers than your actual customers. And, there aren't the access concerns, right? You can kind of, have a little more freedom to use various tools like user interviews and others too, to go find them. So tell us about your process and what worked well, or maybe some challenges you had.

[00:23:08] Matthew: Yeah. Yeah. So, well, the big challenge I had was like for a recent research project. What I, we really wanted to target new users to the Braze platform. And with the goal of understanding how intuitive is our platform kind of just kind of benchmarking our usability overall. But as I said, we have a small pool of customers, so trying to find even newer customers to our platform is harder.

And also you don't really want to bombard the new customers immediately with research on their onboarding to our platform. It's a little sensitive at times. So, you know, having a little, so I have a lot of trouble with this, but it was super important for this study to get this. You don't want it to get people who are experienced and, you know, are able to just quickly find what they know how to do because of muscle memory. So ended up recruiting look alike. I don't know a better way to say, look like Braze customers. And in this example, it was marketers who are familiar with life cycle marketing and some other tools that you can use for lifecycle marketing, like some of our competitors maybe, but that were unfamiliar with Braze as a platform.

And so we use that as a proxy for new Braze customers. So maybe they've used or have used one of our competitors' tools. Maybe they've heard of Braze, but they've never actually used it. So I went about this the longest, what took the longest was trying to figure out the screener for this,

um, 

[00:24:33] Erin: Do you find these people?

[00:24:35] Matthew: Yeah. So I asked a few articulation questions to truly see if they fit our recruitment profile.

Some questions along the lines of like, just trying to think back now since it's been awhile, but like what's describing a typical day at your job. So like, what is your typical day at your job also? What's your job title and responsibilities because there's the field of lifecycle marketing or growth marketing? I'm not one, so I don't want to misspeak, but it feels fairly new.

So there's a lot of different titles out there. So I didn't want to give a long list of titles. I wanted to see what people would, you know, what are people calling themselves and then asking some questions about. What platforms they use or currently use, like I said and they could have selected any, like some of our competitors I threw in some competitors I threw in some that aren't competitors, just to kind of throw people off the track to make sure they don't, you know, like they're not just trying to fit in the system. And one thing I actually did too, and this was kind of fun as you know, all these platforms have such interesting names, you know, like MailChimp, for example. So I just made up a random name and put it like a, made up a random platform. And so if they had selected that they would automatically get rejected from the study 

[00:25:53] Erin: That sounds like a, that sounds like a fun SaaS party game. Let's make a new SaaS app, which one's real.

[00:26:00] Matthew: Yeah, the name I came up with, I literally just no reason rhyme or I just wrote blimey,

which just 

[00:26:07] Erin: Yeah. That could be 

[00:26:08] Matthew: something. Yeah. Was like,

[00:26:10] JH: Did you put the vowels in it or did you,

[00:26:12] Erin: Yeah.

2004. JJ.

[00:26:14] JH: I know.

[00:26:15] Matthew: blimey.io or something.

Uh, yeah. 

[00:26:17] Erin: Yeah, legit.

[00:26:18] Matthew: So that was one way. And if they also, if they had used Braze, I would reject them and yeah, we had pretty amazing results.

[00:26:25] Erin: You could put them in another panel. Those Braze folks. You need those.

[00:26:29] Matthew: totally. I know. It's

[00:26:30] Erin: Save 

[00:26:30] Matthew: other.

maybe they don't use Braze anymore, but they are using them now or either, or totally. But this was great. Like we got the turnaround time was like less than 24 hours and we were able to set up research sessions with everyone that fit our criteria within a week.

And we got some really valuable feedback that we wouldn't have gotten had we just relied on our customers alone. So this is something we're trying to roll out more broadly. Because I think you get a lot of internal bias always just talking to your customers. So being able to talk to people outside, whether they're prospects or whether they're not prospects, they're just using another platform and they are in this role of a marketer is a yeah.

[00:27:14] Erin: They don't know their prospects.

[00:27:17] Matthew: Yeah. 

[00:27:17] Erin: Yeah.

Everyone is a prospect. Yeah. Yeah. I'm curious to hear more about how you think now that you have all these different tools, these different ways of finding who you want to talk to when you're designing a study. And I'd love to learn more about the kinds of research you're doing in terms of, you know, do you have.

Ongoing studies or is it all kind of project and ad hoc based, but depending on what kind of study you're doing, do you have a framework for, I'm going to use these tools to recruit for this, or these are the kinds of people I want to talk to, and therefore I'm going to go through a CSM or I'm going to use a Braze in-app intercept, or I'm going to try user interviews.

Do you have a kind of sense of how all these different recruiting tool kits can fit together depending on what you're trying. To learn and who you're trying to learn it from.

[00:28:04] Matthew: Yeah, so we're developing it. We're in the process. So. It's right now, there's only two researchers at Braze me and one other Sophia. She's wonderful. So we're working together on this. I'm trying to grow the research team and grow our practice, like coming up with good processes. So this is a little bit of a work in progress, but the general rule is, you know, if it's something that's like you don't have to be too specific about in an app, you know, an intercept, like you said, in that message.

It can be a way to go. If it's something that you need a little more you need to talk to more specific users. That is where, you know, you kind of need to dig around in, one of our tools to find the customers that are using the feature that you need to you want to talk about, and then you may have to go through your success managers to contact them directly.

That's probably one of the harder ones. 

[00:28:57] Erin: And do you work with a BI or data team to kind of find those folks or? Yeah.

[00:29:01] Matthew: Yeah, we do. And we have, it's pretty, self-service on our part too. Like, you know, they've made it, they built ways for us to easily look up these things on our own, so we can kind of find those customers, but if we need help yeah. We have a whole BI team that is super helpful there.

[00:29:19] JH: Was this a question I had just from the earlier thing of like, you know, trying to find, not actual users to talk to and using our platform to do so. It made me kind of wonder if you've ever tried to split the difference. Yeah. Going through the sales team and like, hey who are some of the prospects that we lost, you know what I mean?

And could I try to reach out to them? They, I don't know what kind of response rate you get. I don't know if it'd be good because they want to tell you all the things that, you know, they needed that you didn't have, or if they're just gonna ignore you because they went with a different solution, but have you ever tried to like to play in that category at all?

[00:29:46] Matthew: No, I have not yet. That is something I think might be specific to any project we might work on. I would imagine, and I can't say for sure, but I imagine like our sales team for people that they've lost deals to probably do some sort of survey to find out why already. So unless there's a specific reason I haven't done that so far, but if there were a reason to, based on a research project definitely would be a possibility.

[00:30:11] Erin: Yeah. 

to JH's point too, I wonder about, you know, the kind of hybrid project where you talked about this a little bit with using user interviews where basically you can't, it's hard to find the people like you would gladly talk to your own customers if they met their criteria you were looking for and you could find them, but if you can't, then you kind of expand right.

And use some other tactics. I think that happens a lot. It's my sense. I don't know how to quantify it, but

[00:30:35] Matthew: Yeah. Yeah. That definitely in that particular case with the project, that definitely was the case. Like I, you know, was using, I had some new users to Braze, but I paired them with customers who were not new users, but non-users but you know, kind of hybrid mismatch that way, but they generally, they looked very much alike, you know, especially with the goal of the project, which was, you know, measuring, usability, intuitiveness of our platform overall.

[00:31:05] JH: Yeah. Another other question coming to mind is we're kind of getting into the details here when you are reaching out to your own users. So you work with BI and analysts to pull it, you get the sign-off from account managers. Are you just going straight to scheduling with that group because it's a limited pool and you know that they're going to be a pretty good fit or are you still actually having like a screening step where you're collecting some information from them and winnowing it down further from there?

[00:31:26] Matthew: Yeah. Yeah. It's a great question. I think Some people do it differently. Like some project managers or excuse me, product managers and designers have done screeners before, just to make sure that they're getting at the right customers. I generally avoid more friction. Like what I've done is actually allowed people to sign up using Calendly and asking questions in Calendly. To see if they fit the criteria I'm looking for. I wish Calendly had like a screener survey built into that automatically, but so far for me, I've looked through that and tried to target from the get-go on the types of customers I want. So I don't have to add another, unnecessary step of screening them out, but we have done that.

Other people have done that. And Yeah. If there, I would love for there to be an easier way. So there's not a screener that populates the Calendly, you know? So it's less friction. So to answer your question, typically, I go straight to the scheduling and try to target ahead of time as much as possible.

[00:32:32] Erin: So, I know you have experienced B2B rather than B2C the math to B2B. So, you know, across the BS and the CS. But what are some of, we talked about this a little bit just by virtue of talking about B2B specifically, but some of the main differences and recruiting that you've that you've noted.

[00:32:48] Matthew: Yeah. I would say recruiting in general, it's more difficult in B2B than B to C. There are more things to think about, I guess. And just because it's a numbers game, just because there's more. You know, users out there in B to C than there are for B to B. And so all the things I talked about of ways to get around are the reasons why it's more complicated.

And I would say in B2B compared to B to C, it's easier to let bias creep into your recruiting. So we touched on some of this, like over contacting participants. Reaching out to the same ones over and over for different studies. So they become professional testers. They're more likely, possibly more likely to give you start saying things you want to hear.

So over contacting them, using the same participants over and over those are probably like the things that are easier to happen with B2B. I think it's B to C. You can, because you have that number, you can put more guardrails around that like you have this group of customers.

Once they've been contacted and don't contact us again for another six months, we could do that on our B2B platform. But with the amount of research that we're doing, I, we quickly would run out of people, I think. So.

[00:34:04] JH: How do you think about getting the right balance for the, like, following up with the same people? Right. Because it actually feels like a tough problem, I found somebody that does this behavior or uses the product in a way that's relevant to what I'm trying to learn. They're really articulate.

They're interested. They're willing to talk to me like they're checking all these boxes. You probably have a session with them. That's great and informative. And you get all these interesting takeaways. How do you know when it's okay to follow back up with that person? Like what, how do you have any rules of thumb or, you know, is it just a time thing?

Is it what you're trying to learn? How do you get that right?

[00:34:35] Matthew: Yeah, I think it's more of what you're trying to learn than a time thing. You know, if you're trying to do something, that's a participatory design project where you're building a feature and it might make sense to have someone who. It is like the same person giving you feedback round after round.

In some cases they're kind of co-creating this with you and not just one person, but a multiple group, as long as you're kind of also hearing from others. So it's not just a singular voice. I think that's fine. And so I think there's pros and cons to both, and I'm a fan of both. So in that example and you know, time to the super engaged people are easier to recruit and our product Braze.

It does require some experience to use. So there are some people who are really good at using our platform and know everything about it. So they're good people to tap into and get feedback. And, I was talking about bias before or maybe over contacting the same people.

And they're more likely to say what you want to hear. I haven't found that so far at Braze. I think our customers, if they have an issue, they're going to let us know if they like something they're going to let us know, but if they have an issue they're gonna, they're gonna let us know. So I think it's fine.

I do think, you know, the con of talking to the same people over and over is the possibility of confirmation bias creeping in. And so the pros of having new voices, you know, in your projects. I think that's most likely obvious to researchers, you know, removing that bias and making sure you have a you know, healthy mix of people that are telling you, not just what you want to hear.

So, that's kind of a non-answer. I think there's pros and cons. It's not an either or to me, I think there's pros and cons to both. I'm a fan of both. And really depends on the type of project that you're working on too.

[00:36:22] Erin: Yeah, it makes a lot of sense. And I think too, I mean, do you have a customer advisory board? I know you mentioned the bonfires that's like, those are different things or.

[00:36:30] Matthew: Yeah, I would say we don't necessarily have a customer advisory board. We have a bonfire that's our community. And so I would say that is kind of, you know, our de facto one,

[00:36:41] Erin: Yeah. So, so through that community, or just from people who you kind of informally end up talking to on a repeat basis, to your point, you know, bias can, if that was all you were doing, that would probably not be ideal, but on the marketing side, You know, those people who stick with you and are invested in the platform and your success and are going to give you good, honest feedback.

I mean, that's so invaluable in terms of just building advocacy and word of mouth. And as long as. The folks in that kind of circle remain, you know, your ideal customer and the people that you want to get more customers that look like them. Right? As long as that audience is kind of maturing with the platform, there's so many benefits there in addition.

So it's like everything in research, right? You need a balance. You need a hedge portfolio.

[00:37:30] Matthew: Yeah, absolutely. I definitely agree with that. And yeah, we have other ways that people input, you know, information, if people have feature requests You know, we have a way for people to do that and those get prioritized and we might reach out to customers based on the number of feature requests we hear about.

So that's another way we gather information. That's just kind of as an aside, but to your point, yeah, it's all about.

[00:37:54] JH: I'm just curious. You've been, you know, you've been doing this for about a year now. You're, you know, we're going into 20 22. So you're probably looking ahead and doing some planning, anything that you're going to try to change or explore or try some new things when it comes to B2B recruiting going forward.

[00:38:09] Matthew: Yeah. Well, we'll see where the world is, but I definitely want to put a few more processes in place. And along with my colleagues and. Just to make it, so it's not just like, we're only talking to the same users repeatedly that we know it should be a balance.

So a few more processes in place on this and maybe some sort of processes about which customers, when have they been contacted last, maybe we don't set a time limit necessarily on not contacting them again for another six months, just for a number of reasons. We know when they were contacted last and or when they were interviewed last and to make sure that we keep that in mind.

So, this is not just research. This is kind of a group thing that will involve our success team and everyone just to centralize that information. So we're not over-communicating other customers. So that's one thing I want to do, just more operational. Another thing I'd love to do.

And again, this just depends on what the world looks like, but I would, you know, I think a lot of researchers probably would love to do this, but like more, you know, in-person research and little observational research. We'll see how that goes. So that's one thing I'd love to do is I think getting certain people who are harder to reach, like the developers, for example, would love to be able to like shadow developers as they're onboarding Braze and integrating with Braze and learning about that. Like in real time, rather, you know, I think one of the things with remote is we've often just been relying a lot of times on, you know, customer interviews or concept testing and observational research. We could have diary studies obviously.

But I would love to just be able to observe in real time again. So that's something I've looked forward to, but you know, we'll see where the world goes. Everything.

[00:40:00] JH: Yeah, no, that makes sense.

[00:40:01] Erin: Matt. Thanks so much for joining us. What final parting thoughts?

[00:40:07] Matthew: Yeah. I think. I would love, you know, hopefully this is useful for people who are thinking about going to maybe a SaaS company or you're, you know, the researchers who are going to be working in B2B and that this is useful, or for people currently working and to see if this resonates with them.

And maybe also I'd love to hear from people who you're like, no, what about this and stuff I didn't think about. So, Because it's definitely not something I thought about when I joined a B2B company. So definitely. And it's fun and it's challenging and it can, I think it's worthwhile. So hopefully this is a useful thing for people.

[00:40:46] Erin: Yeah. Thanks Matt I'm sure that it will be. And to your point about, you know, other tips and things people have tried, are you active on LinkedIn or Twitter or anywhere else? We can kind of keep the dialogue going.

[00:40:56] Matthew: Yeah. LinkedIn, mostly.

Um, 

[00:40:59] Erin: so it can find Matt there and we'll post this up there and see what other tips folks have.

[00:41:04] Matthew: Cool.

That sounds great. 

‍