The State of User Research 2022 report has arrived! Explore the data.
SUBSCRIBE TO OUR NEWSLETTER
September 26, 2019
Step-by-step how I ran my first card sort and all the lessons and learnings along the way.
As with so many things, the first step is to Google. I’ve been part of card sorts before, helping other teammates update some information architecture, and for launching new brands and websites working with brand teams who are experts on this sort of thing. But this was my first time running one on my own. Like many researchers—and I am not a researcher, I just play the part sometimes, perhaps like some of you—I was short on time and resources. Still, I knew the value of validating my ideas with my intended audience.
I read about open and closed card sorts and determined open would be better for my needs. I had a long, but not complete or final, list of topics I knew we wanted to cover. I built these from an internal survey focused on our customer questions and needs, browsing respected websites in our domain, and seeing what’s popular and how they organized things, etc. I did not group them or put them into any sort of hierarchy. I had ideas, but I really wanted to see what trends emerged from my group of guinea pigs participants.
I’m excited that we are launching a Bring Your Own Audience product soon. (Write JH@userinterviews.com if you’re interested in providing feedback!). This will allow me to communicate with prospective study participants through our platform. In the meantime, I used our marketing automation tool, Autopilot, to reach out to our audience of researchers who had launched a project in the last year. I sent them a quick text based email:
I was thrilled with the response. It’s wonderful to have such an engaged audience eager to participate in user research on the “other side of the table.” 53 people completed my card sort and 59 started it. The average time to complete was 21:29. I can see all that here in the tool I used, Proven By Users.
Speaking of Proven By Users, I came across this tool doing some searches on things like “remote card sorting tool” and after pretty quickly comparing a few options, I determined Proven By Users best met my criteria for this card sort: ease of use, quality and variety of analysis tools, cost: free, thank you beta program!
I definitely recommend it. Check out this dendrogram action. I confess, first timer on the dendrogram. Add that to the learning list. Score.
After spending some time with the various reports—similarity matrix, maximum agreement, groups, cards, groups + cards—the loudest signals I arrived at were:
The result of these and some internal kicking around is a structure that looks like this:
We’ll end up with 30-40 lessons in our initial plan, likely to build and iterate from there. We’re looking forward to a February soft launch of the first few modules, and you’ll be able to sign up to get the full experience, or individual modules of interest, as an email series. We’re really excited about it and determined to make it a wonderful source of essential UX research content. Make sure to sign up for our newsletter (top or bottom nav form) to be notified when it launches!
Proven By Users lets you include a survey. I asked a question soliciting general feedback on the card sort itself, because learning. Some highlights:
I had no idea how to categorize the cards, so any sort of prompt would have been helpful (unless part of it was to see how we would sort the cards).
Yes and no! I certainly could have provided more context even though point was to see how folks would organize cards on their own. I worry about providing too much up front context and losing folks sometimes, but I’ll definitely provide more in the future.
I wanted to group analysis with something else. But it's really its own thing.
Agreed. Would love to have done group, in-person sessions, but was able to get a lot of feedback quickly here. Next time!
Terrible! There was no (obvious) way to get context for sort?!?!?
More context. Got it!
What do you learn by forcing me to sort all cards? I should have been able to leave some cards in the list, especially the cards that have no value or meaning to me.
I probably could have better explained that not every card had to be sorted! I think I was relying on the app to handle that more clearly than it did.
Interesting worthwhile venture.
Despite some useful but critical feedback on the card sort, I was happy to see an 80% open rate on my thank you note and 0 people who opted out for ongoing communication. You’re a great bunch!
After I publish this blog post, I’ll share it out with the new segment of awesome participants who completed the card sort, to keep them (you) informed on our progress.
We’ve already begun designing the experience and crafting the initial content, and I’ll share the progress of those efforts soon.
Tactical and strategic skills are great, but a “soft skill” I’m also learning to be more comfortable with is vulnerability. User Interviews is building a strong culture that values this through consistent 360 feedback, monthly personal growth team sessions, daily water coolers and a general vibe of humanity.
There’s this great New York Times article I always remember about how discomfort actually helps you live longer. It’s kind of kooky, but I love the idea. Fundamentally, learning is about embracing the discomfort of not knowing, working through that, and emerging bigger.
So in my new job and in this new year especially, I hope to keep embracing that discomfort and to continue learning and growing. Thanks for joining me on the journey.
Want to contribute to User Interviews content? Here’s how.
VP, Growth & Marketing
Left brained, right brained. Customer and user advocate. Writer and editor. Lifelong learner. Strong opinions, weakly held.