Best Practices

thoughtbot’s Jaclyn Perrone on Preparing for Amazing Research Sessions in Any Situation

Research sessions don’t always go according to plan, but having backup plans and a user focused attitude will take you far

Carrie Boyd
December 5, 2018

What do you do when your user research session doesn’t go according to plan? It happens all the time, in a lot of ways, for a lot of reasons. Fortunately we have Jaclyn Perrone, Design Director at thoughtbot, on the pod today to talk us through it all. She shares stories of times things didn’t go quite right, what to do when participants get stuck on the wrong things, and how to avoid (avoidable) mishaps.

Episode Highlights

[1:07] Jaclyn talks about her role at thoughtbot and her UX career journey.

[4:05] The gang talks about how things go wrong when you do things with other humans (like user research)

[4:43] Jaclyn talks about what happens when research goes wrong and what to do without WiFi

[7:06] Jaclyn answers the question, “Is it more important to be prepared or ready to improvise?

[8:21] How to deal when the prototype breaks

[9:36] How much can you really add to the test anyway?

[10:24] Test the test

[11:52] What to tell your participants to expect during research

[12:07] Jaclyn warns against the dangers of fake data

[14:10] How to positively boost your participants confidence, even when they focus on the wrong things

[14:53] What to do when you’re the note-taker and the facilitator strays from the script

[18:31] How to deal with leading questions and how to get over the fear of asking a leading question.

[20:58] Jaclyn’s top tips for avoiding research mistakes

[22:29] How to be more comfortable asking stupid questions

[24:08] Structuring better scripts and making your participants feel more comfortable

Plan Ahead: Foresee Problems and Prevent Them

Test the Test

Testing the test is what it sounds like. Once you’ve completed your test design, walk through your test before showing it to real participants. Best case scenario is you can get a co-worker or someone less familiar with your test to try it out. You’ll almost always find something you’d like more information on, a question that is worded in a confusing way, or a broken link in your prototype. There’s nothing worse than being mid-study and realizing your test is flawed in some way you could have easily changed ahead of time. Measure twice, cut once.

Have backup plans for things you can foresee going wrong

One benefit of testing the test is it can reveal weak spots or potential hurdles. Having a backup plan can help you stay calm when things go south and continue your research with minimal speed bumps. Jaclyn, for instance, ran into a testing situation where she didn’t have internet or data to run her live usability tests in the field. She ended up pulling up some Sketch files and showing her prototypes that way, but left the research a little sad that she couldn't show her files the way she had wanted to. If you’re doing field testing that requires an internet connection, consider getting a hotspot device that can keep you connected wherever you are. You could also consider hosting the files you need locally, or creating paper printouts in case of emergency.

If you’re working with electronics of any kind, pack some extra chargers—if you’re doing in-person testing, bring extra for participants. If possible, having a backup device you could do your test on in case of an emergency is a great safety net too.

Count on a few no-shows or less than perfect participants by creating a small backup list of participants. Ideally you won’t need them, but it’s certainly possible that you will.

Don’t Put Unrealistic Fake Data In Your Test

Nothing throws research participants off more than distracting, unbelievable data. Take a look at this contact card.

First off, the picture makes it pretty hard to focus on anything else. While it’s good for a quick laugh, it makes it difficult for your participant to focus on the task at hand. When testing with a prototype that requires written content, it’s best to take some time and try to make things as realistic as possible. This gives you an opportunity to gain some insight on information you may be missing and how the final product actually will work for the user.

Jaclyn also says Lorem Ipsum is a major no-no in prototypes It’s not realistic and it won’t give you a good idea of spacing, or of how the customer would react to the content you’re thinking about putting on the page.

Improv Time: When Research Sessions Don’t Go Perfectly

Don’t Let Interviews Stress You Out

If it’s your first time conducting user interviews, you’re probably pretty nervous about messing it up. The good news is, the people you’re interviewing are also human, and there’s really not much to be afraid of. It’s just you, a person, talking to another person.

If you ask questions that are leading, or violate other best practices, just note it, try to get the insight you need another way if it’s really important, or simply move on.

Get comfortable with awkward silences 😅. You’ll run into them a lot when doing user interviews and, if you wait them out, they can usually offer you some good insights! Don’t feel rushed to give examples after you ask a question to get a quicker answer or fill the silence faster. Letting your participant come up with something on their own will get you where you’re going faster.  

Use your script, but keep it loose, more like an outline. This allows you to improvise a little and make the interview more natural. It also leaves room for you to learn things you may not have with a more formal script and  you’ll stress less about getting through a set number of questions, even if they don’t fit in the interview. Focus on hitting your main learning objectives, and keep it flexible beyond that.

Don’t Make Assumptions

Check your ego at the door. Conducting research is for learning about your users. Don’t be afraid to look (a little) stupid or ask stupid questions. Your participant may have a completely unique way of doing something you think is fairly standard.

Research is for learning and finding new ways to grow your product and business. Keep that humility at the front of your mind and you’ll be less stressed by minor mishaps and more focused on what really matters.

About our Guest

Jaclyn Perrone is a Design Director at thoughtbot. She’s also an artist, musician, and all-around awesome person. She hosts a podcast at thoughtbot centered around digital product design.

P.S. Can we email you?

Get "Fresh Views," our weekly newsletter, delivered to your inbox. We feature interviews and point-of-views from UXers, product managers, and research nerds of all stripes.

Carrie Boyd

Carrie Boyd is a Content Creator at User Interviews. She loves writing, traveling, and learning new things. You can typically find her hunched over her computer with a cup of coffee the size of her face.

Recent Posts

Subscribe to FRESH VIEWS newsletter

Weekly interviews and point of views all about UX research

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.