You’ll learn when it’s best to use generative interviews in your research and how to conduct effective interviews
Research methods can be divided into two phases: evaluative and generative. Evaluative research is used when you’ve already developed a product, and you’re trying to evaluate what’s working and what’s not. Generative research is used to—you guessed it—generate ideas. These two types of research are not mutually exclusive. One can support the other throughout the early stages of development. Generative research techniques include generative interviews, which is what we’ll cover here.
In a generative interview, you’ll interview people with the purpose of uncovering opportunities for innovation and for illuminating solutions to problems. Generative interviews allow you to build up a nice bed of rich information about your user base from which you can pull ideas, both for brand new project builds or improvements to existing products.
Generative interviews are all about the discovery phase, when you still don’t really know what problem you’re trying to solve. You might have a general idea about what a problem is, in which case generative interviews can help you refine your understanding. Or, you may simply want to develop a product in a given space, and you need to generate ideas about what problems exist before you can imagine their solutions.
Without generative interviews, you could build a product no one wants, or develop a solution to a problem that doesn’t exist, or misunderstand a problem just significantly enough to make the solution essentially useless.
Generative interviews are used early in the design and development process when you’re looking for opportunities and ideas. But they’re not simply brainstorming sessions. They are used to gather the information you need to understand potential users and their lives well enough that you can design a product they will actually want and be able to use.
This channel of communication—the channel between you, your users, your colleagues, and your stakeholders—is one that you’ll want to open early and keep open throughout the lifecycle of your product. The earlier and more in-depth the conversation with your users, the better chances your product will have for long-term success.
Generative interviews kickstart a project by involving key players right from the get-go: users, stakeholders, anyone who might have an impact on the project one way or another. One interviewee might lead you to the next, and in this way, you build a community and a network around the seedlings of your project as it’s being born. By involving others in this way, at this stage, you’ll spark interest, curiosity, and hopefully some level of interest or commitment. People are compelled toward the conversation and the project, and will hopefully become invested in its success alongside you.
Asking for an interview also communicates your interest in and respect for the interviewee. When you need information from stakeholders or from other important allies, asking for an interview—rather than some less personal data-gathering technique—is a good way to start. Everybody likes to be treated like an expert.
Another primary advantage of interviewing is its simplicity; you want to know what a group of people think and feel about something? Just ask them.
Interviews are less expensive, and much less time-consuming, than observational field studies; but, if conducted in an open-ended way, are just as likely to give you valuable information you didn't think to ask for.
Failing to do appropriate generative research can lead to wasting time on a product no one wants. Consider the long list of products throughout history that failed because, as the saying goes, they were great solutions to problems nobody had. Alternatively, a product can fail because it does not take into account the realities of its user’s life.
A hypothetical example would be a fifteen dollar book entitled "How to Survive on Less than Five Dollars a Day." Nobody who needs such a book can possibly afford to buy it. Silly cautionary tales aside, if you’re not already personally familiar with the context in which you hope to design, chances are you'll miss an important variable somewhere. Which makes generative research essential to your process.
The pitfalls of generative interviews are the same as the challenges with interviewing in general; the data can be more subjective than a more observational approach like a field study and less quantifiable than a survey. For these reasons, it usually makes sense to combine several types of research for one study. That said, all things considered, user interview data is some of the best qualitative data you can get.
Things to be aware of: interviewees, being human, can misremember. fabricate, or become fanciful in their recollections or opinionating. A person might leave out important information because they take it for granted that the interviewer either already knows or does not need to know. Even a simple question, like "What do you want?" might not have a straightforward answer, since not everybody knows what they want, even within the simplest contexts.
The shortcomings of interviewers can come into play as well. An interview is a relationship, and if the interviewer cannot hold up their end of the dialog to direct the conversation, or if they can’t effectively build rapport, that relationship is not going to work, even throughout the short duration of an interview.
Non-leading questions are essential in the generative phase. If the interviewer isn’t able to put their own dogma aside and enter into an interview with an open mind, in a neutral state, they won’t elicit the information they need from the interviewee. Likewise, combine qualitative approaches like generative interviews with quantitative approaches to fill in any holes.
When you start out with formal usability testing, like I did, you learn to moderate by serving it up cold. You treat every participant the exact same way—using the same words, even the same facial expressions and body language, if you can manage it.
I tried that—and I couldn’t manage it. It didn’t feel right, and it didn’t feel like me. So I tried something different: building a rapport with my participants. Asking about the weather or about their commute before diving in. And I started to get something I hadn’t gotten before: more feedback than I asked for. More than just a response, a story about how what we were focused on in the test fit into the larger picture of the participant’s lives.
I still believe this is the key to getting honest, organic data, but I admit that when I first discovered this, I went too far. I dialed up the warmth, the interest, and the open body language too high. I developed such a strong rapport with a participant that a session focused on his college research habits turned into him asking me to grab a beer after the session!
I realized my mistake. Flustered, I muttered something about research ethics and sent him his gift card with an impersonal “Thanks again!” It was a good lesson for me in how much is too much, and how to balance genuine warmth with rigor and ethical practice. I now also usually bring a co-researcher to sessions with me, just in case things get weird on either side.
The process of conducting an interview is more than just talking to an interviewee. There are stages of planning and preparation beforehand, and stages of documentation and analysis to go through afterwards. Even when interviews are more casual, or take place between colleagues who know each other well, it helps to be clear and focused on your purpose.
Things to consider in the planning stages are:
And so on.
Generative interviews often generate ideas for who to interview next. So, keep an open mind and if one interview leads you into some unexpected territory, go for it until you feel you’ve hit the point of diminishing returns or run out of time or budget.
“What would you do if…” can be a useful question, but in many cases, it’s not as useful as “How did you handle it when…”. It’s a golden rule in job interviews, too: past behavior is the best indication of future behavior. So begin in real life scenarios where the subject has prior experience.
Generative interviews can be open ended and glorious in how they meander and illuminate areas you might not have even considered. Still, generative interviews can suffer from interviewer assumptions or bias. To keep yourself accountable for your own presumptions and preconceived ideas, try to keep questions a bit more vague, and aligned around words like how, why, and what—open ended, in other words. This will guide your interviewee toward their own answers, rather than leading or prompting them toward the areas where you want or expect them to go.
It helps to come in with some idea of what you think is true, but generative interviews are only useful if you allow ideas to be generated. So without being overly leading, go in with an idea, and don’t be overly attached to your rightness or wrongness. Just let the conversation move you toward or away from this preconceived notion.
But feel free to deviate from it. You don’t want the interview to feel stilted. You want it to feel loose and have a flow, and you want your interviewee comfortable. That said, if it’s too loose, you could find yourself all over the map. So conceive of a questionnaire ahead of time to guide the conversation without keeping you penned into to a certain conversation. If the interview is taking a useful turn, follow it where it wants to go.
If you identify a handful (3-6) of larger topics you’d like to cover, and identify a handful of subtopics that step from each large topic, you’ve got yourself a nice loose guideline for an interview that won’t be overly confining. For example, say your area of interest is yummy snacks. You might conceive of larger topics like cookies, popcorn, charcuterie, and healthy snacks. And beneath each main topic, you might have a few subtopics. In the end, your list of topics might look like...
Cookies: gooey, crunchy, chips, nuts, fruit
Popcorn: traditional, savory, sweet
Antipasto: meats, cheese, peppers, olives
Healthy snacks: fresh fruit, dried fruit, nuts
And you can use this list of presumptions as a guideline for understanding how, why, and what your interviewee looks for in a yummy snack (if that’s indeed what you’re trying to learn) without locking yourself into a prescribed list of questions.
In some cases, the number of interviews may be obvious; if you’re able to identify fifteen people from the company organization chart who might have information you want, you can plan to speak with all of them. Much more often, there are thousands or even millions of potential interviewees, and you have to decide how to pick representative individuals in a number that makes sense.
There is something of an art to deciding how many people to interview. In general, the more people you talk to, the more information you will get, but only up to a point. Eventually, the interviews get repetitive, and interviewing more people won't teach you anything new. If you notice that you have reached that point, you can go ahead and cancel any more interviews you have scheduled, but for budgeting purposes, it is better to know ahead of time. You will have to make the calculation case-by-case because no two studies are exactly the same.
Start with the number five, since you are almost certain to need that many, and add or multiply based on your needs:
Note that your interviewees do not have to be strangers to your team. A good interview technique can draw out information that the interviewees don't realize they have, so you can go ahead and interview stakeholders and even team-members in order to get all your resources and all your project parameters out on the table.
Recording an interview can mean anything from taking handwritten notes to videotaping the interview. If you use notes, use teams of two to conduct the interviews, one to ask questions, the other to take notes, if you can. Remember that if you use audio or video recording, you’ll want to budget time and money for transcription.
Once you have your plans in place, it’s time to do your interview!
Does your equipment work? Is your computer charged? Are your proverbial pencils sharpened? Cross all those Ts. Fill a bowl with candy or snacks. Dress the part. Show up on time.
Begin by introducing yourself. Briefly explain the reason for the interview and how long you expect it to take—your interviewee may already know all of this, but just recap. Warm them up with some chatty questions. Ask them if they have any questions of their own before you begin.
Save more personal questions for later in the interview as well, after you have built some rapport. Feel free to ask chatty or slightly off-topic questions, especially at the beginning, in order to help your interviewee get warmed up, especially if your topic is something he or she might not be thinking much about otherwise. Allow time to get focused.
Watch for signs that your interviewee may be getting tired or uncomfortable. Digressions or apparently excessive storytelling can be great opportunities to learn things you didn't think to ask about, and allowing some digression is good for rapport, but be prepared to diplomatically pivot back on topic when necessary. Do not let an interview go into overtime, since your interviewee has places to be—as do you. You can always schedule a follow-up if need be.
Wrap up gently.
Finalize with something thoughtful like, “I just have one more question before we wrap up,” or “To wind things down, I have just one or two more questions.” This will make the end more gentle, especially if you have someone really pouring their heart out.
Even after you have asked your last question, stay in interview mode and keep your recording device on until after you’ve said your goodbyes and left the office or hung up the phone. You want to remain professional, friendly, and courteous, and you also want to be ready in case your interviewee decides to say "just one more thing." It's not uncommon for important insights to be offered as your interviewee is on their way out the door.
Documentation and analysis are critical aspects of any type of research.
Documentation in this case is really about getting your interview data into shape to be shared and understood by other people, even long after the fact. Recordings should be transcribed. Shorthand should be written out into full, clear sentences. Any details jotted down in the margins should be pulled out and organized and placed into context in notes. Even if you’re relying on recordings, write down thoughts and impressions and add those notes to the transcripts. Some study designs call for giving interviewees a chance to review the transcripts, notes, and initial analysis, in case they have any corrections or retractions. Place all your records, including copies of the recordings or field notes and any amendments offered by the interviewee, together for easy later retrieval.
As for analysis, there are various forms of qualitative data analysis relevant to generative interviews, from the simple and casual to the involved and rigorous. You may use a two-stage analysis: first looking at each interview in isolation, and then looking for patterns across interviews.
Once your analysis is complete, write up your results in a complete report for later use. While part of the benefit of generative interview involves the learning process of the interviewer personally, you also want any member of your team, including stakeholders, to be able to gain insight from the interviews.
Interviews can be combined with other methods, such as observational field studies or focus groups. An initial series of interviews can lay the groundwork for a field study, or an observational field study can develop insights that are later used in planning an interview series. Participant-observer studies often include interviews.
The same recruitment methods, tools, and logistical planning—and sometimes even the same interviewees—can be used in both the generative and evaluative phases of a project. In the generative stage, you might pick their brains regarding the pain points of using a certain product, and later ask them to use the product to determine whether those problems have been resolved.
The logistical demands of an interview series depend largely on where the interviews take place. If you invite interviewees into your office or lab, there is little for you to do beyond scheduling and showing up prepared. Travel, on the other hand, requires more time and money to plan.
If you're doing your interviews remotely, find a web conference interface you like and get familiar with it ahead of time. Stay tuned for a future chapter on all our favorite user research tools!
As for tools, the most important tools you’ll need will be note-taking instruments—either your computer, or a good old fashioned notepad and pen. You might also decide to use a recording device of some kind.
The best way get your subject to be at ease, and to conduct a comfortable and productive interview, is to be at ease yourself. Get them going, capture their ideas, and sort it all out later with your colleagues in the analysis process. Remember that most people don’t know what they know, just as most people don’t know what they don’t know. It might be difficult to articulate what an ideal tool might look like to accomplish a certain task, for example, but in a good conversation the nuances of this ideal tool could become evident through questions and answers and meander and reveal other truths. So be a great listener. Drop breadcrumbs for them, and likewise follow those unconscious conversational breadcrumbs they’ll likely scatter. Look for clues, pay attention, and have fun. You could be mining the next big thing.