uxdesign.cc – User Experience Design — Medium | Caitria O’Neill

On-the-go debriefing methods to help your team make sense of the research they just observed.

A bit of humor goes a long way after a full day of research.

You’re tired. You just interviewed 7 people back-to-back in a small room lined with one-way glass. You’re already starting to lose track of who said what. Your team is in the observation room, hangry and quickly losing focus.

This is a perfect time for a debrief.

Why Debrief?

Debriefing is discussing the research that just took place with your team or clients. This can be as formal as a moderated activity, or as informal as a conversation over a team dinner. In general, it is best to run debriefs as quickly as possible after running an observed research session.

• To Learn: having team members who observed the study share their main takeaways can reveal novel insights. Knowing how your team interprets the research they observe can give you a sense of what was most important to each stakeholder, and can help you address misunderstandings head on.
• To Build Consensus: discussing core findings together can allow team members to embrace new findings and new product direction, without feeling as if the previous idea ‘loses’.
• To Triage: using a debrief to break research insights into immediately actionable vs. bigger issues can help team members begin forming a plan of action.

This article will share three common formats for working with stakeholders to debrief research findings, and a handful of conversation topics to get you started.

Three Debrief Formats

1. MODERATE A CONVERSATION

Debriefs are best conducted immediately after your group observes research and the sessions are still fresh in their minds. This will sometimes mean you’re conducting a debrief over dinner, or in the back of a taxi in Mumbai.

Moderated conversations are flexible enough to take place over dinner, but still allow you to steer discussion towards the most important elements of the research and next steps.

Your goal is to help your group explore WHAT HAPPENED during the research session, and WHAT IT MIGHT MEAN. This helps the group move past superficial insights (ex. participant didn’t like the button ) to actionable insights (ex. people were worried about leaving the site).

TIPS FOR A GOOD DEBRIEF CONVERSATION:

• Set ground rules: explain the format of the discussion. Set expectations around the types of contributions you’re looking for, and your role as moderator.
• Set aside decisions: debriefing is time to talk about all possible interpretations of the research that was just observed. Gently nudge the group away from trying to definitively answer questions, or debate around the best way to ‘solve’ a problem. Listen to ideas, but move people on.
• Take notes: the discussion should reveal what your stakeholders find most important, and can help you decide how to frame and focus the scope of your findings. You can use this perspective in your report to expand upon the findings the team finds most surprising, or informative.

2. WHITEBOARD TOGETHER

Assembling your team around a whiteboard helps you build a common understanding together. Generating and sorting post-its, or adding

Whiteboard debriefing sessions should also move you from sourcing reflections WHAT HAPPENED and discussing WHAT IT MIGHT MEAN for the user or product.

STRUCTURING A WHITEBOARD DEBRIEF

• Post-its: post-its are a very flexible way of structuring a good debrief. You can introduce a series of prompts, and encourage Use additional prompts to have the team re-shuffle these reflections by severity of issue, or ease of addressing.
• Group: putting thoughts on post-its allow you to organize by concept or theme. For example, you can have your group generate reflections on each participant, one thought per post-it. Then have the group pull out the post-its with themes that appear across several participants and create a new cluster.
• Rank: in a good debriefing session, you’ll generate a lot of insights, and potentially lists of priorities or immediate next steps. Use the flexibility of post-it notes to do a quick ranking activity at the end of each section. For example, have the group sort a list of usability problems by severity, or the ease of the technical fix required.
• Set aside ideas and disagreements: when debriefing what happened, sometimes it is tempting to jump immediately to what “should be done” to fix it. To keep the group focused on exploring insights rather than framing solutions, put proposed ideas on a post-it note in an Idea Parking Lot off to the side, for later follow-up.
• Page-Per-Participant: for each participant, tape a large sheet of paper to the wall. Have your team add observations and insights as each interview is conducted.
• Worksheet: You can also ask team members to fill out a worksheet for each participant, to help structure their observation and analysis. Solicit these reflections ‘popcorn-style’, by going through the list of participants and having your team call out what struck them most.
• Divide and Conquer: assign notetakers for each participant as the study is taking place. At end of the day, each notetaker shares their observations about the participant they covered. After reviewing ‘what happened’, use another sheet to track common themes, issues and opportunities that crop up across participants.

3. Communicate Asynchronously

Development at Facebook moves at lightning speed, and most people work on several projects simultaneously. That means you’ll often only be able to get part of your team into the observation room for user research. You also may not be able to include your team as observers in international field work.

Lightning speed development can also mean that decisions need to be made before your complete analysis is available. Rather than waiting for a polished presentation, you can start introducing your findings to your remote team and encouraging discussion while the research is actually happening.

You can choose whatever sharing and discussion platform works best for your team. At Facebook we used Facebook posts to share details about ongoing research sessions with our colleagues. Thoughtful photos can help people empathize remotely, and useful discussion takes place in a comments section. Some researchers created Facebook Groups to share photos and insights from international trips with team members who stayed behind.

Asynchronous debriefing does take some time — remember, you’re doing a first pass of analysis, sharing it with the team, and asking questions to inspire discussion. Plan time between participants to post updates, or a once-a-day cadence for field research.

IDEAS FOR ASYNC DEBRIEFING

• Post Participant Highlights: choose a set of key questions to help structure highlights for each participant. How did the participant react to each? Try to streamline highlights to keep your team focused on the most important insights from each session. For example, rather than relating verbatim what happens, share the biggest surprises or contradictions.
• Ask Questions: discussions don’t always start themselves. Including direct questions about the material you’re sharing can help draw people into discussion. On Facebook we tag team members in posts to alert them to questions that relate to their work specifically.
• Debrief As Soon As Possible: asynchronous sharing does not replace a good face-to-face conversation or whiteboarding activity. Because asynchronous sharing can be mostly “broadcast”, you don’t receive the same amount of feedback on what information team members find most important. You also have less influence over how people interpret events.

Topics and Activities

Ultimately, the structure of your debrief is less important than the content. Here are a few debrief discussion topics from Facebook researchers to help you get started.

• Traffic Light Activity: “I find that the traffic light activity can be pretty effective. Have people individually organize their thoughts into keep doing this (green), stop doing this/fix this (red), TBD/there are some questions (yellow). We make space for green, yellow and red on a board and have people tack on their post-its accordingly.” — Jen Shuang
• Top 5’s: “We whiteboard it with post-its. Top 5 insights, Top 5 next steps, chart out impact/effort of next steps. I’ve had pretty good feedback from XFN partners about these meetings.” — Laura Roth Rivera
• Mini-debrief, Big Debrief: “Quick debrief with team members after each session/participant for 15 minutes. In the course of the day, and these mini-debriefs, we look at both 1) low hanging fruit, short-term changes we can make (start tasks, come up with design solutions, start to observe patterns). 2) A bigger debrief at the end of the day, where we go over quick-wins, low-hanging fruit versus bigger issues to fix, opportunities etc.” — Jill Campaiola
• Round Robin Research Objectives: “I like to structure discussions, round robin style for several rounds. In the first round, I ask what the single biggest take away for each of the team members was. That helps me understand where they are and what they believe is the most important thing they learned. Then, we usually have several rounds of going through each research objective, round robin again.” — Anja Dinhopl
• The Dinner Debrief: “Casual dinner debrief discussions after a day’s research with the end goal of coming away with one new product implementation we’d like to design or test has also produced actionable and acted-on results.” — Liz Keneski
• Media Show + Tell: “Actively sharing themes and leading the conversation. I focused on Sharing Inspiring Stories by showing the artifacts (collages) and sharing pictures, stories, quotes, which generated a lot of spontaneous discussion.” — Jessica Peterson
• Least Surprising Findings: “I have one last round of questions, which is, what was the least surprising learning coming out of this research? That helps team members feel like, hey, I really know this and feel confident in this direction or finding now.” — Anja Dinhopl
• Postcards from the field: “each observer creates a postcard back to the team (who were unable to attend) — draw a picture and a main insight and why it’s important to know.” — Sin Lee Loh
• Matrix: If you’re trying to help the group work through how an experience felt, you can make a matrix and use it to organize reflections. Create a 2×3 or 3×3 matrix and label axes with relevant levers like “Pain” and “Delight” or “Useful” and “Gets in the way”. — Sin Lee Loh
• Journey Map: Collaboratively sketch out the user’s journey through the app. What actions do they take, and how is the experience? Where do they mess up? What feels delightful? Push your team to connect their contributions back to things they observed in the sessions. — Sin Lee Loh

Do you have additional debrief discussion topics or activities? Please share them in the comments below!

Republished resource collection — big thanks to researchers Anja Dinhopl, Jen Shuang, Laura Roth Rivera, Liz Keneski, Jill Campaiola + Jessica Peterson for their collaboration on the original article.

Discussing research results with your UX team was originally published in UX Design Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.