SUBSCRIBE TO OUR NEWSLETTER
Katryna stopped by the podcast to talk survey design, the impact of stakeholder engagement on researcher fulfillment, and medieval big data.
[8:20] Thinking about ways to improve survey distribution for a more diverse audience—and the implications that has on how representative that makes the results.
[11:40] On average, researchers rated their fulfillment at work as 6.4/10. How does stakeholder buy-in and engagement with UX research affect that score?
[18:51] A staggering 90% of user researchers said they worked exclusively remotely since the pandemic began. How has that affected research?
[27:00] How to get stakeholders engaged with research by involving them early and often.
[28:34] 34% of people who do research start planning their research sessions 2-4 weeks in advance. We ask: Will we see that timeline shrink in the future?
[32:20] Survey nerds need to know about the Domesday Survey of 1086, and Katryna’s here to tell you about it.
*Psst—Katryna here… Bonus extra fun fact that I forgot to mention: In the 11th century, many rents were paid in kind (i.e. not with money). One of the most common forms of payment was eels. Yes, as in the slippery fish. Domesday data shows that people in England paid their landlords over 500,000 eels each year. 🐟🐍
Katryna Balboni is a content marketer by day, thankless servant to cats Frodo and Elaine Benes by night. Loves to travel, has a terrible sense of direction. Bakes a mean chocolate tart, makes a mediocre cup of coffee. "Eclectic."
Katryna: [00:00:00] the people that were most fulfilled were folks who said that most or all of their stakeholders know how to access, research findings and do so at least some of the time. So there's obviously a relationship there between, you know, how much you feel your organization, your stakeholders, your team values, the work you're doing. I mean, that's not shocking.
But also how often are people actually accessing the insights that you're generating through your research?
Erin: [00:00:37] Hello everybody. And welcome back to awkward silences today. We're here with Katryna Balboni and she is our Content Director here at User Interviews. So thanks so much for joining us, Katryna.
Katryna: [00:00:50] Yeah, I'm excited to be here a little nervous as well, but yeah, very excited.
Erin: [00:00:57] happens to the best of us. We've got JH here too.
JH: [00:01:00] Yeah, no need to be nervous. We'll go hard on you.
Erin: [00:01:03] Yeah. And so we're here to, we brought Katryna in today to torture her. Cause that's what we like to do to folks. But really to talk about our third annual state of user research report, which we affectionately call sour, turning lemons into lemonade. And yeah, so this is our third year bigger and better than ever before.
And also Katryna's first year being here to steer the ship. So Katryna after your first year of, you know, putting the survey together, finding participants to take the survey, collecting the data, pulling together the insights. Putting together the report, what jumped out to you, you know, as you're kind of acclimating to this space of UX research and getting to know our audience uh, what jumps out to you as being really interesting in this year's report.
Katryna: [00:01:55] Yeah. I joined in October and we started putting the survey together in December. So it definitely, you know, it was part of that learning process for me. And one of the big things that jumped out was definitely, it is impossible to design a perfect survey. And in some ways I enjoyed the challenges and I'll talk through how you know, we approach some of those, but the format definitely leaves a lot to be desired.
And then on the insight side of things one of the things I found really interesting was that the lack of buy-in and stakeholder access to research was the thing that had the biggest negative impact on job fulfillment compared to other frustrations. But this is also something that is. Not easily solved, but it is solvable.
And so I wanted to talk through some of the solutions that we have for that. And then finally, you know, it's impossible to sort of talk about 2020 and user research without talking about the fact that user researchers were not working in their normal environments. So what are some of the changes that we saw in the survey as a result of COVID and everybody going remote and where do we think that's going to take you?
So those are some of the, I want to talk about today.
JH: [00:02:57] Cool. Erin, I'm curious from your perspective on, we've done this a few years now, like year over year, like. How is the state of the union? So to speak, it seemed like user research is heading in a good direction and you see a lot of the same frustrations each year. Anything stands out kind of year over year.
Erin: [00:03:13] Yeah, well, to Katryna's point, it's hard to sort of longitudinally compare this year to last year, given that we had a global pandemic
JH: [00:03:22] Yeah. This year is, yeah. It's such an outlier. Yeah.
Erin: [00:03:24] But that being said, a lot of us were very fortunate in tech, in, you know, these kinds of knowledge, digital jobs, where you're able to, not all of us, but a lot of us largely kind of adapt and keep our jobs.
So in that sense, you know, UX research kept on. Kept on humming. And in that sense in fact, the companies were doing more research than they were doing before. So in a lot of ways it made an actually, you know, hard to say it, but positive impact in some ways on research, I would say generally a pandemic aside, if you can even say that The state of the union is strong and, but it's not, it's a two steps forward one step back thing, I think.
Where some progress is faster than others. So one area where I think there's been a good amount of discussion and lip service is around accessibility and inclusion and diversity. And that was an area where I think I'm going to be looking in future years. To see a lot more meaningful progress because it's obviously one thing to say that those things are important.
And it's another to see people say, and we're doing something about it. And I think those numbers are not where we want to see them. But in terms of more people doing research earlier and more often you know, getting by and getting budget, a lot of those things are really moving in the right direction. So Katryna, let's jump in. Tell us, where do you want to start? You want to talk about, you want to get meta? You want to talk about the survey design itself? We at User Interviews, of course think participants are the center of the universe. So who took this thing? And what did we ask them about?
Katryna: [00:05:04] Yeah, so, I can get meta. So who took this? I guess I'll start with distribution. So who were we? How do we reach people? And we of course distributed in our own newsletter on our social channels, but also in communities on LinkedIn, Slack, Facebook, places where user researchers congregate, we don't own those channels.
And we ended up with sort of a 50, 50 split which is great because we don't just want, you know, to be surveying our own customers and, you know, announce a hundred percent of user researchers use User Interviews. And the majority of people.
Barely majority 51% were located in the US and that's another nice even split there because we were able to, in some cases, for some responses to parse out, you know, US researchers and non US researchers.
So when we talk about salaries and things like that and the sample size that we got was 525 folks who do use user research as at least 10% of their job. Most folks do more. And so that's, you know, it's not, it's not a huge number, but definitely enough to talk about some trends in the industry and, you know, to analyze things on a more granular level.
Erin: [00:06:16] Yeah. You know, we made a decision consciously when we launched the V1 of our sour report. We wanted to talk to not just full time researchers, because that really does narrow the field of people who participate in research, who benefit from research. And as democratization of research becomes more and more of a theme, we didn't want to sort of limit insights from only people who only do research. At the same time, we didn't necessarily want to overstate you know, insights from folks who occasionally or rarely or never do research either. So that's kind of where we landed in terms of that balance.
JH: [00:07:01] Yeah. I mean, like when I scan through the stuff in the report, it is a really impressive kind of mix of different cross sections of backgrounds and locations and experience and all sorts of factors. I'm just curious. Do we think you mentioned having some things to improve for the next time around. Anything around like how we syndicate or distribute the survey to kind of continue to improve, you know, the region and who we talked to.
Katryna: [00:07:24] Well, yeah, so that's actually, I sort of was thinking about this and like, it gets to be almost like a philosophical question. So, let me backtrack a moment. So for, in terms of like company, diversity of company sizes, we had a pretty good mix again.
You know, we had 50% of folks around 50% work in companies of 500 and fewer. And then you have 25, 21% of people who work in companies with more than 10,000 employees, which is mind boggling to me, someone who has spent her career in startups. So on that front and you know, we have a good mix.
Most of the people that we heard from are individual contributors, 75% folks. And then I think 16% were managers, so that. Gives us a pretty good sense of or, those folks are able to talk about, this is what I'm doing in my day to day, but then of course we don't necessarily always get the insights about, what is the budget for research and maybe on the higher organizational level.
So maybe some rooms for improvement there, but one thing. That, in terms of diversity that I'm really interested in improving, is the representation in terms of like, who, who demographically do we reach and who are we talking to? And so most of the folks that we heard from are identify as white and in the US 74% of us respondents identified as white.
And then of course, people could opt not to respond at all. And we had 6% identify as Black or African-American. And with the sample size that we had, that's a really small number of researchers. So it's hard to then get into the data and say anything really meaningfully about, you know, how things might, how racial or gender identities might impact things like salary or fulfillment at work, et cetera.
And so as we think about how are we going to distribute next year's survey? You know, we do, like I said we distributed this in Slack and LinkedIn and Facebook communities, but very general ones. So we could make a much more concerted effort to send this survey out, to user researchers of India, Facebook group, or you know, folks who are uh, Black folks in UX.
And then we would get more representation for, for different demographics. But then I was like, Oh, well, does that mean that is that actually representative of like the field, because you know, tech in is unfortunately a very white field currently. At least in the US like that's the reality.
So, yeah. You know, are we getting a representative sample? But of course, for most of the questions we're asking, it's, whether you are Black or Latino or white, you know, it's not going to impact like how many diary studies you do regularly. So it's for things where those demographic questions are really important, like access to healthcare, you know, anything like that.
I think. Being very cognizant of trying to balance out where you're distributing is important, but for us that's less of a concern. And certainly we want to hear from a wider variety of voices not just racial and ethnic and gender identity, but you know, across the board. So I think finding those sort of niche, I guess you could call it communities and promoting it is something that I'd like to do more of.
Erin: [00:10:51] Yeah, I know one of the things we wanted to lean into in the survey was really job satisfaction and how happy or effective do people feel? Because, and again, you know, in this very normal of years I think all of that. You know, the idea that you have like a work life and a personal life and like that you can sort of compartmentalize is really been made challenging and in some bad ways and in some good ways But obviously you know, being happy at work is very important to everyone who is spending 40 plus hours of their time at work who wants to get meaning, right?
Like this is like the big millennial career stories. Like these crazy millennials want to actually enjoy their jobs, you know, what's up with that. And so we wanted to, you know, lean into those kinds of questions in terms of, is this work fulfilling for the people that do it. And what did we find there that was interesting?
Katryna: [00:11:44] Uh, Yeah, so this year, so we ask people to rate their, sort of, fulfillment at work on a scale of one to 10, which, there's no getting around the fact that this is like a very subjective rating. And each person interprets that differently, but the average score was 6.3 out of 10, which is. Point three points lower than last year, which considering the 2020 we had, it feels kind of like a win.
But what was interesting was that the biggest differentiator in terms of how, you know, if somebody, when you parsed out, like who is more satisfied or less satisfied or fulfilled, less fulfilled at work, the people that were most fulfilled were folks who said that most or all of their stakeholders know how to access, research findings and do so at least some of the time.
Um, So these were folks who gave us an average of seven out of 10 rating compared to 5.5 out of 10. So it's a 1.5 difference. Folks who said the stakeholders would never access retreat research findings. And similarly the lowest score among all frustrations we asked, what was your biggest frustration at work?
The lowest score came from folks who said that lack of buy-in about the importance of research was their biggest frustration. So there's obviously a relationship there between, you know, how much you feel your organization, your stakeholders, your team values, the work you're doing. I mean, that's not shocking.
Uh, But also how often are people actually accessing the insights that you're generating through your research?
JH: [00:13:13] It makes me realize, as we were talking about like, who takes the survey, there's a component or a segment here of people who decide to leave the field, right. Who are like, this is so frustrating. And you know, whether it be because of internal buy-in or, you know, it is, I think we've had other guests talk about, you know, the kind of toll it can take on you as the researcher kind of, you know, Being on the front lines and hearing other people's struggles and stuff, depending on the research you're doing.
So I don't know if that belongs as a component of this survey or if that's a whole separate investigation of like, almost like an exit thing of, you know, people who were former user researchers and actually gave it up for these reasons. That I don't have a coherent thought here. Just like, feels like a very interesting population to understand.
Cause I bet there's a lot of people with some strong opinions in there. That'd be cool to learn from.
Katryna: [00:13:54] Yeah, and we definitely did not capture those opinions because one of the screener questions, you know, it was a requirement that people are currently participating in doing research in, you know, in their work now. So we don't have those insights.
JH: [00:14:08] Right, and for the, for the purposes of this survey, in terms of the trends that we're trying to share, right. And present everyone, I think that's appropriate, but there is this kind of like survivorship bias of the people who are most dissatisfied don't take the survey they, peaced out.
Katryna: [00:14:19] Yeah. And also, I mean, it's also like we are hearing another thing like, Oh, when we talk about X, percent of companies are doing this, it's also, we're talking about companies that do have people doing user research or do have dedicated user researchers. And so that's not necessarily representative of, you know, whatever industry we're talking about because not all companies do have somebody doing that. Unfortunately.
Erin: [00:14:42] Yeah. And then another reason we wanted to include folks who don't necessarily do research full-time because it really does eliminate a ton of companies who are trying to do research, but maybe you don't yet have a full-time person. Cool. What else did we learn in this year's state of the user research report?
Katryna: [00:15:00] uh, Yeah, so a couple or I thought it was fascinating. And this is just sort of, there's not a whole lot to dig into here, but I was. Amazed that 70% of the people that we heard from have master's degrees or higher which is, that's just a very highly educated group of people.
But 45% only 45% have degrees in human re scent. Sorry. Human computer interaction or UX. So, you know, related degrees, basically.
Erin: [00:15:29] Yeah, that'll be that'll be an interesting one to watch too, in the years to come where. When you look at the different undergraduate degrees that folks have the different paths to UX research, they're like all over the place. Some interesting ones do come up. Anthropology, psychology, architecture pretty much all of the humanities, a lot of behavioral science, et cetera, et cetera, but really everything.
But part of that is that there haven't been these direct pathways into UX research and you are starting to see more of that where HCI has been around for a while. But like UX research degrees, dedicated UX, masters degrees, a lot of those are really in their infancy or, you know, being augmented by bootcamps or online learning or these like sort of alternative modalities of learning.
So I think that will be really interesting to see if that becomes. More structured or as in, you know, the last year where, you know, learning and online learning is really evolving into something more modern and flexible. Maybe that path will continue where you actually, you know, see a doubling down of these kinds of alternative, more flexible ways of learning UX research.
Katryna: [00:16:36] Yeah, certainly. And new roles too well, not new, but emerging fields like research ops is something that we touch on very briefly in the report and have written a few pieces on. I think it was actually on this podcast. Kate Towsey research ops guru, essentially.
She made the argument that once companies reach, eight folks doing research on a regular basis, you've really sort of reached this threshold where you need opera operationalization. That is such a tricky word. You need an ops function in order to continue doing efficient, effective research at scale.
And based on the responses from our survey, it seems like companies are starting to reach that number of researchers around like the 500 to a thousand employee mark. Obviously that's not like you don't reach that and suddenly, desperate need for research ops, but that is sort of a good benchmark to be aware of.
JH: [00:18:17] Results. Like, obviously there was a huge shift in 2020 to everyone working remotely. And we see that in the data very clearly. It seems like, and I haven't seen the raw results. I'm curious to get your take on this. It seems like a lot of the things that were hard for researchers. In terms of working remotely, we're kind of the things that were hard for anyone working remotely, right.
In terms of being connected with teammates and how you collaborate, work and life blending together, all that kind of stuff. Was there anything that stood out around remote work and user research that was kind of like unique, if that makes sense or or surprised you.
Katryna: [00:18:51] I mean, yeah, a lot of it isn't truly surprising. I think if you're somebody who works behind a computer um, there's, you know, and is able to work from home. There were a lot of similarities in the experiences that people have had this past year, but certainly, the fact that researchers need to conduct research is, is unique and, and what it means to do remote research versus in-person research and how, you know, Remote research is great in a lot of ways.
And we are certainly huge fans of it because it can be, a lot faster, it can be easier for certain types of research. And especially now that Video conferencing tools are like second nature to so many people. And it allows you to recruit from a much broader geographic range or people that you might not normally be able to bring in for an in-person session.
But then, you know, not being able to do in-person research for things that really might have a very tactile elementary you're not, there are. It's forms of research that you just can't do remotely well. And I think that's a frustration that is, unique to this, the group of people that are talking to for this.
Erin: [00:19:59] Yeah, it'll be interesting to see what innovations come out and physical products given that we've like supplanted our entire physical lives with digital experiences and how the two come together, how would you like for, to fire physical experiences more with a digital layer, but you know, as researchers have been forced to move so much research to remote just be interesting to see what, what our lives look like when we return to the world of the living.
Uh, So thinking ahead, right to presumably sour, will go on to see another year. Have you started to think about what that might look like next year?
Katryna: [00:20:37] Yeah. So as I talked about, you know, thinking already about like, how can we distribute this more mindfully, but also, you know, it should be, I, like I said, I was really still finding my sea legs when we launched this report or sorry, this the survey and there's a lot of stuff that I've learned already since then that, certain questions that I like to ask specifics about what methods people are using and also doing a better job.
I think of. Mapping out the tools landscape that people are using currently or the tools people are using currently to offer a better, more accurate list for people to choose from, you know, like last year, for instance, Miro and mural were both write-ins and this year 60% of people said that they used Miro for organizing their notes.
And obviously, I mean, That's a huge growth, even accounting for the fact that they were write-ins. And that does point, I think, to the adoption of remote tools that are designed to facilitate remote collaboration, obviously in this unusual work environment that people found themselves in.
But. It is hard to, you know, say that super definitively because they were, they were write-ins and we didn't have that as an official option. And there were some things this year as well, that were write-ins and, would more people have said, yes, I use this tool if we had it listed.
So I think doing a little bit more research upfront about. When we have these options, making sure that what we're presenting is really an accurate list of things so that nothing gets left out and we're not misrepresenting the popularity of certain tools or approaches, et cetera.
Erin: [00:22:15] Yeah, there's always the irony. You talked a little bit about the, you know, the limitations of surveys in general and there is no perfect survey. One of our most popular, awkward silences episodes I believe is called why surveys almost always suck. So there's always this kind of irony whenever we're running a survey of like, how do we make this like a really good one?
And so you talked about some of the ways we can do that. Have you thought about fortifying the survey with other research methods or really expanding what the state of user research can be.
Katryna: [00:22:46] Yeah, I mean, I haven't honestly, I haven't thought about like every way that we can expand this. Certainly though. I mean, it's a long survey and actually this year ended up being longer than last year is although we had a higher completion rate. And I think a lot of that had to do with, you know, we, we made a very.
We took a lot of time in the beginning to look back at last year's and see where people were dropping off and essentially making sure that it was optimized to flow more logically from one question to the next. And I think there's a lot more we can do there as well too, I think, mix it up uh, or make it more engaging.
I'm not that's a goal of mine. That's not a strategy I don't yet have, I'm not yet sure how we're going to do that while still maintaining the integrity of the data. But yeah there's a lot to do, but because there's so many questions that we want to ask, but then also we need to like, be very conscious of, Well, when do we, are we going to just completely lose people?
What is too much? And I think we're kind of already there. So how can we get the insights we want without overwhelming or
Erin: [00:23:48] Yeah. Sometimes, less is more.
JH: [00:23:52] we ever done user research on people who took the survey, like following up with them about their experience of going through it and especially the people who dropped off. Like, I wonder if there's an interesting kind of exercise there to unpack some perceptions from folks of, if they, you know, maybe some people thought it was short, you know what I mean?
Or like, whatever
Erin: [00:24:08] Yeah, or do like the meta, like, Ooh, the exit intent, like actually, would you like to take a survey on why you quit taking your survey? But like to your point, JH about finding the people who are no longer in the industry, we gotta, you know, follow up with. I know we do follow up with everyone who kind of took it the previous year and see if they want to re-up on taking this year's survey.
But it would be interesting to try to find those folks who have left the industry, but could we know what we started to talk a little bit about one of your. Favorite insights. If one can have a favorite insight around that level of my insights equally, right? Good or bad. The most insightful of insights uh, around you know, this fulfillment being related to stakeholders, knowing where insights are using them.
Right. And that to me has. Sour aside and sour really like kind of evaluated or validating. You know, what I've kind of noticed happening anecdotally, which is that is the thing, right? Where, you know, having been in this role for over three years now, like things do change over three, three years a pandemic aside, and it felt very much three years ago.
Like the conversation was around. We need to be doing more research and like that was happening. And, you know, you're seeing more research roles happen, research ops, starting to begin as a thing. And obviously there are people at companies that are still having those fights to justify needing research, needing the budget at all, let alone, you know, from project to project, but it does feel like the conversation has evolved largely to, okay.
So like we're doing research. But is anyone using it? How do we really measure the impact of that research we're doing? How do we prioritize our research projects? And a lot of those answers really hinged around, well, what is it being used for? And a lot of that comes down to, well, how quickly can you do it?
Right? Which is not to say that all research needs to be done at breakneck speed. Right, but that there is this value of good researchers research that gets used. Is that something that you've noticed as well?
Katryna: [00:26:10] Yeah, I mean, well, so, there's an article on the blog six ways to do faster user research without sacrificing validity and really what that's about is not like just shaving off, you know, seconds from the actual research process itself, but research sort of long term. Because what you're really getting at is you want insights faster.
You want to be able to surface the insights that you need to make better product decisions, et cetera. And. I think that also is very, or it is very closely related to, you know, do people know where to go for these research findings? Like can people access the research that you're doing? And so one of the things that we say is, you know, it's so important to create a culture of research and building up that long-term organizational knowledge.
And a huge part of that of course is making sure that the people that need to access your research can and know how to so you know, that involves one way you can do that is by building a better research repository. If you don't have one, make one, and if you have one, make sure that it's working for you.
And then another way that, a very important thing that speaks to the frustration that a lot of user researchers are clearly feeling about stakeholders, not being bought into the importance of research to begin with. Or not knowing how to access the insights that they were hoping to get from whatever research they bought into is to make sure that you are involving your stakeholders early and often.
Of course, doing interviews, stakeholder interviews at the beginning of the process to learn what their goals are. Not so that you can, then. Doctor the results are when you give them, but to sort of, you know, present it in a way that speaks to the answers, their questions and that's, you know, going to really make their ears perk up and also to allow them to participate in research you know, be active, but non-disruptive observers summarize observations, present them to non researchers.
And have them sort of own the translating those insights into actionable recommendations for their departments. I think through that process, people really get a better sense of like, okay, this is actually applicable. I see how, you know, the importance of it really becomes clear when people start interacting with the research that you're doing.
And I think that's that's something that we'd love to see more of now that more people are doing research in general.
JH: [00:28:29] Yeah, one related to this. And one of the trends I was most surprised by, or not surprised by it, but I guess maybe curious to see how it evolves in coming years was how far in advance did you start planning your typical research? And it still is super common for it to be like over two weeks, right.
Two to four weeks is really when you look at the chart spikes up and it makes sense. And if you're in like a good momentum space, that's probably fine. Cause you're just rolling and always doing research. But I know Basel, our CEO will sometimes bring up, like if you can't, if a question comes up and you're like debating it and then someone's like, well, have we talked to users?
And you're no. And it's like, If you see that as being three or four weeks out, like you're just not going to, it's like, it's not going to factor into that decision. Whereas if you see it as a couple of days out, or, you know, a week out, like there's probably gonna be chances where you can fit it in. And so I'm curious to see if that will change over time or organizations will get a little bit more nimble with how they're able to manage that and plan or if just the way it works is like, it is kind of a rolling thing.
And the next one's always a couple of weeks out and we just are always doing it. So be super curious what responses look like.
Erin: [00:29:27] that would be an interesting one to segment out by the kind of research people are doing too, because if you're doing like in-depth discovery research and you know, I don't think anyone feels like that's the kind of research that. You can't plan, you might say like, what do we want to learn this quarter?
What do we want to learn this year to be planning for it depending on your company and what kind of timeframe you're thinking about, right. If you're Apple, let's say you're planning years and years ahead, and you're making big bets, right? Big, expensive bets. Obviously that's going to require some planning, but we know that a lot, the majority of research that people are doing that is.
The easiest to justify and kind of every kind of company is the quicker, like, should we do this? Are we on the right track? And that's where that speed is really important. Like no research should take longer than it needs to. Right. But two weeks is certainly okay sometimes. Right. But there are times where two weeks means not going to happen, not going to get used.
So I do think it would be really interesting to look at the timelines in terms of. From planning to recruiting, from planning to doing, from planning to using the insights. How many times did those insights get used? People probably don't know that. Right. But maybe they would like to know that really depends on the goal of the research and the kind of the research.
That would be really interesting to look at it, I think.
JH: [00:30:53] Yeah, a lot of factors you can't separate out, but it still seems the amount of people who say a few days ago or like, a week ago still seems really low to me. So I'm
Erin: [00:30:58] Yeah no, absolutely. Absolutely.
Katryna: [00:31:01] I think also the fact that it's so easy to access quantitative data too when you're like, Oh, I can just, people that aren't, that aren't necessarily, fully convinced how important. Qualitative data is, they're like, Oh, I can, I can talk to a thousand people I can put, or I can pull up data on a thousand people right now.
Why would I wait or invest this time in hearing from five if individuals? So I
Erin: [00:31:24] and it depends on the question. And I think that's, to me, that's the ideal state of where the democratization of research winds up is where more people get, build the muscle for, or have access to a researcher who can help them figure out how do I answer this question? What's the best way to answer this question in the time that I have, is it a quantitative answer with a tool we already have?
Is it talking to five users tomorrow? What you can usually do through User Interviews, shameless plug. Is this like a strategic discovery question of really redefining who our audience even is and what they care about and the, you know, so it's really understanding what methods, but also what timeframe and budget makes sense for the question you're trying to ask.
JH: [00:32:09] True. True. As we were prepping for this Katryna, we talked about your wealth of medieval facts. And, I think I promised that we would bring it up on the pod. So, I think you have some info about a very old survey. Is that correct?
Katryna: [00:32:20] Yeah. So, I mean, This is, feels like my moment. Like I'm a failed historian and I have a captive audience, and I want to talk about so many things. Like, I want to tell you about medieval animal trials and such. But I won't instead I will tell you that if you're interested in surveys and the history of surveys or just.
Want to be sort of blown away by a really amazing example of like medieval big data collection. Then you should look into the doomsday book of 1086 and aside from having like a really metal name it's just an amazing administrative accomplishment. So the background of it is, and I'm not an expert.
My area of study was the late medieval period, a little bit of high medieval which this is not. So I will link to a podcast on this subject, in the write up of this podcast for anyone that wants to dive a little further, but quick plug it's a, it was a survey that was done in 10 86. 20 years after the Norman conquest of England.
And for folks that don't remember their AP history class this was when the Normans who were in France came over to England. There was a whole long complicated, you know, inheritance that was behind this. Anyway, William, the conqueror, took over England 20 years later. He wanted to know how much his conquest is worth essentially.
So he, or probably some smart folks in his administration came up with the domesday survey and. It was actually called this because it was like so thorough that you couldn't escape it, so like the judgment at doomsday which is kind of funny, but what it was is he had his people go all throughout England with the exception of a few counties in the North to document how property was divided, how much a property was worth and what taxes were owed.
So like really just taking inventory of how much was this fertile land going to bring him in in taxes, but no survey approaching the scope and extent of this was attempted in Britain until almost 700 years later. Like it was a huge thing. And what's really interesting is that in addition to like the raw data collection, then you essentially get like an analysis of the data and examining it in various different ways.
And all of this was happening on parchment by a few individuals. So it's just really interesting and, you know, I can't provide a whole lot more information, this isn't the place, but I will link to the podcast for folks who want to learn more.
JH: [00:34:48] original census basically.
Erin: [00:34:49] yeah.
Katryna: [00:34:50] yeah, yes. Quite a census and there were censuses before this at various points in history.
But yeah, an enormous land survey,
JH: [00:34:59] I'm imagining somebody sitting around in a room with piles of parchment and like manually counting stuff up. And then somebody comes in and interrupts them with a question and there's like, shit, I have to start
it back. I don't know what number I was at. Like.
Erin: [00:35:09] Yeah. Yeah,
Katryna: [00:35:11] I mean that, that probably happened. It's when you actually are studying these old manuscripts, you see that like it's very
JH: [00:35:18] two, three, four. You're like
Katryna: [00:35:21] And then also you get like a cat walking across to see like cat prints and ink and stuff. Like all the same things that distract us today. Just a little bit more basic formats.
Erin: [00:35:33] thanks Katryna Thanks for joining us.
JH: [00:35:35] This is fun.
Carrie Boyd is a UXR content wiz, formerly at User Interviews. She loves writing, traveling, and learning new things. You can typically find her hunched over her computer with a cup of coffee the size of her face.