By Veronika Sipeeva (opens in new tab)
Photo credit: iStock
Researchers have access to a great number of UX evaluation methods, yet testing a design with 5-6 users seems to be the most common response to research requests in small and large organizations. However, not all designs or prototypes are ready for or deserve to be put in front of our customers. In fact, many problems with an experience can be uncovered by running a heuristic evaluation or conducting an experience review, as has been covered in this previous blog post (opens in new tab).
Evaluating a design against a chosen set of heuristics often helps scope and shape the focus of a researcher’s next steps. And in some cases, a “heuristic-evaluation-first” approach yields feedback that convinces teams to re-work the designs before conducting studies with customers. But while heuristic evaluations are less expensive, they’re typically not as exciting to run as usability studies or interviews, and they don’t produce convincing videos for stakeholders.
So, how can we raise the stakes for heuristic evaluations and when should we?
Bringing a remote team together
The idea came when I was asked to test a set of customer journeys. Initially, I wanted to approach this by running a quick heuristic evaluation before conducting a usability study. But with end-to-end journeys each spanning a multitude of pages, it would have been impractical for one person to tackle.
Instead, I decided to bring in in-house evaluators. At that time, my team members and I were all working from home due to the COVID-19 pandemic. Had we been in the office, we could book a conference room, bring in some lunch, and go over the UI Tenets and Traps cards together. But with everyone so busy and immersed in their work, I needed to find a way to entice people to spend time on a generally not-so-fun activity.
It’s time for a heuristic evaluation party
I marketed the effort as a party and posted a call to sign up on our Teams channels, adding a fun gif to bring attention and lighten this task.
Instead of inviting only expert evaluators, I extended the invitation to everyone interested in helping and learning about this technique. I used this as an opportunity to bring people together and help them meet someone new or someone they had never worked with before. It was a perfect way to get our colleagues familiar with this method, to grow the next generation of evaluators, and potentially to offload myself from this kind of research in the future. For more about this, see our prior blog post: Give your research skills away: Why coaching product teams won’t put you out of a job. (opens in new tab)
As a result, a multi-disciplinary team of researchers, UX writers, and designers joined our parties. To maintain the party theme, I assigned my colleagues to groups based on musical instruments e.g., guitar, piano, etc. I paired people who were familiar with heuristics and people who were using them for the first time.
Aside from the agenda, each party opened with a customer quote and a video recording from one of the relevant studies. Then I introduced the method, heuristics, or rather UI and UX tenets and traps, and our reasons for choosing them. I also had a demo-practice together before each pair left to an assigned Teams room to work on a specific part of the journey.
Each party guest had a set of heuristics to focus on and a reference card with questions to guide them through. The journeys we evaluated were put together in PowerPoint, so we used this medium for capturing issues. When someone found a problem, they would just add a shape or an icon to indicate the area of the problem on a page and attach a comment with an explanation.
When this effort was completed, we had held six one-hour parties with a total of twenty-seven participants. In that time, we covered a tremendous amount. We knew that we would get a lot done, but we didn’t anticipate it being this powerful at the outset.
When to use heuristic evaluation parties
As with traditional heuristic evaluation, your main goal is still to uncover potential problems with a design. Similarly, you may not have access to fast unmoderated studies or recruitment budget. And finally, you notice that at a glance the design violates the best practices and you expect a usability study won’t uncover any new information.
But there are some additional reasons for choosing a party format:
- You want to teach others how to run a heuristic evaluation and apply it to the future designs.
- You want to get a buy-in from your stakeholders by having them go through evaluations.
- Designs that you are going to evaluate are too large to tackle alone or to get in front of the customer.
- You want to expand your colleagues’ focus beyond the area of their specialization.
- You need to include multiple perspectives and look beyond the UI.
Adapting your practice to your needs
Researchers are generally accustomed to meeting customers in person and collaborating with our teams in the office. In the face of an extended period of working from home, we can adapt our online tools and technologies to emulate some of the in-person constructs and bring back fun and a sense of team connection. It just takes some creativity!
Our first set of parties was lean. But you can take this idea even further, and if budget allows you can order snacks or gift cards to your guests, use fun backgrounds during calls, set up music, create badges or certificates to acknowledge and thank people for the effort. There are so many ways to set a party mood and create a sense of community.
I hope that this post has given you some ideas of how to innovate within your research practice or just within your team.
What do you think? Is a heuristics evaluation party something that would be of value for your research practice? What are ways that you’ve innovated as a result of working from home during the pandemic? Tweet us your thoughts @MicrosoftRI (opens in new tab) or follow us on Facebook (opens in new tab) and join the conversation.
Veronika is a design researcher on the Office Planning & Research team at Microsoft, focusing on the small business space. Previously, she led research projects at a Pacific Northwest health insurance company, Premera Blue Cross, where she helped establish and champion research practices, mentored and equipped teams with tools and methods for improving customer experience. Veronika holds a master’s degree in Human Centered Design and Engineering from the University of Washington, serves on the HCDE Alumni Leadership board and is a co-chair for the volunteer committee for the UXPA International Conference. She is passionate about helping others discover the value of research and strongly believes that the best work often comes out from including diverse perspectives and partnering with others.