By Jessica Tran and John Porter
Note: This article was originally published on January 30, 2020, on Medium
One benefit of joining the UX research community at Microsoft is the chance to participate in the company’s Hackathon, a yearly event that brings employees out of their daily roles to collaborate on new ideas. Our project showed us why partnering closely with people who may use our designs is always best practice.
“Nothing about us, without us” has long been a saying within the disability rights and accessibility communities. For researchers and designers, the motto reminds us to work in community throughout the design process. This notion underpins Microsoft’s accessibility initiatives, including development of the Inclusive Tech Lab. It also drove our team’s process during two years of innovation at the Hackathon.
Our project focused on closing the technology gap that limits options for gamers who are blind or with low vision, who often rely on auditory cues to play. A problem arises for these gamers because the vital social component—laughing, trash talk, yelling “watch out!” to a remote teammate who’s about to get ambushed—usually involves voice chat or the system reading text chat aloud.
If gamers who rely on audio devote attention to the social experience, they’re more likely to miss essential game cues like the furtive footsteps of a sneaky opponent.
To explore this problem space and design a solution, we included blind gamers and designers on our Hackathon team each year, as well as strong UX research representation. Testing our designs and experiencing accessibility barriers alongside our teammates who are blind or have low vision ended up shaping the project in unexpected ways, spurring it to unanticipated heights.
Teamwork at the Hackathon
Our project meant something different to each of us. For Jessica, a UX researcher and engineer, it was a unique opportunity to make something new with accessibility at the core of the design. For John, an intern, it was a chance to leverage primary research he’d done at the University of Washington to identify the unmet need we were exploring. For all of us, the project provided an exciting occasion to work with others who care about accessibility and take a leap into the unknown.
The air buzzed with the sound of more than 18,000 people kick-starting their projects as we brainstormed around our table in the Hackathon tent. Things went smoothly at first. The group hypothesized that a controller with Braille capabilities would allow gamers who rely on audio to communicate during gameplay.
But as soon as we started ideating on designs, we hit a barrier to accessibility for our teammates who are blind: whiteboarding.
In order to be truly inclusive and integrate all stakeholders, we had to embrace a MacGyver spirit in our ideation process. Some of us verbally described sketches being drawn in real time and typed descriptions in OneNote so team members who are blind could use screen readers to hear them. We made it work, but the difficulties of ideating under these constraints pushed us to move toward physical prototypes more quickly, so we could test our designs by putting them in people’s hands.
Building prototypes to test designs
The first question we had was whether a module at the back of an Xbox One controller could allow gamers to comfortably chord Braille cells, essentially typing in Braille. We used a 3D printer to create a prototype with six paddles, corresponding with the six dots in a Braille cell. The gamers on our team who are blind helped us iterate until the position of the paddles allowed them to chord input comfortably.
For reading incoming Braille messages, we envisioned a refreshable Braille display on the back of the controller. The tech gap was too great to build this out at the Hackathon, so we kept this aspect of the prototype low-fi, punching out Braille letters on index cards. We recruited Braille-literate participants from different corners of the tent to test our hypotheses, for example that the Braille cells on the readable display should flow from left to right. When a participant pointed out that this orientation was awkward, we checked our biases and pivoted to a vertical orientation—just as legible but more natural for someone with a controller in hand.
Drawing on our hybrid skill sets, we iterated until we had a prototype that could send messages to a console. When members of Microsoft’s senior leadership team came by, having heard about our project, we were able to hand them a controller that had a story behind it. We told them about the adjustments we’d made so far and why, then offered them a chance to try the input prototype.
Company leadership was clearly excited about our hack, including Chief Accessibility Officer Jenny Lay-Flurrie. When she engaged the paddles, then watched the phrase she chorded appear onscreen, her interested expression transformed into a beaming smile. She told us we were on to something and encouraged us to keep going.
Riding the wave of enthusiasm
The support of Microsoft leadership inspired us to keep working on the project after the first year’s Hackathon. We shared out our findings, talking with experts in different divisions and members of the accessibility community. The process showed us that although we were bringing a new and exciting concept to the field, there was room for further innovation, particularly with our output model.
A refreshable Braille display was still significantly out of reach of current technology. So, how else could someone receive Braille messages? For year two of our Hackathon project, we brainstormed ideas and eventually landed on the concept of haptic feedback.
Our teammates who are blind supported the idea that the same six paddles used to chord Braille could theoretically become a legible mode of output if each paddle vibrated independently.
Of course, we had to test this theory with another prototype, this time with individually vibrating actuators. Some adjustments were needed for different hand sizes, but our blind teammates and other Braille-literate participants confirmed that they could read using the paddles already at their fingertips.
From idea to invention: Closing the loop with patent applications
Drawing on continued support at Microsoft as well as Jessica’s experience as a patent holder, we were able to file patents on our designs that included the three major components we’d tested: a refreshable Braille display, haptic-feedback output, and Braille typing.
The eventual granting of these patents was a satisfying validation that we’d brought a new idea to an unmet need in the gaming space. Although there are no plans to develop the designs into product, the experience of working from foundational research through to invention encouraged us to keep innovating around accessibility, continuing our work to close the tech gap, and connect more people through games.
We’d never have reached this point if not for close partnership with people who are blind or have low vision. It’s no accident that the most accessible aspect of our testing—a prototype providing a sophisticated tactile experience—became the vehicle for socialization that most excited everyone. We see this theme play out again and again in the world of accessible design. The act of inclusion opens new doors for all.
How do partnerships foster innovation and inclusion at your organization? Let us know in the comments below, or tweet Jessica, John, or @MicrosoftRI. Like us on Facebook and the conversation!
Jessica Tran spearheads the growth of internal tools for engineers to connect with customers at scale in product engagement funnels to realize the customer-driven culture at Microsoft. Other research projects of hers include applying the scientific method with big data analysis and customer feedback to understand perceived risk, customer retention, and growth of the Windows Insider Program. Jessica is passionate about women pursuing STEM careers, having earned her Ph.D. in electrical engineering from the University of Washington. She holds five patents (three focused on making gaming more accessible) and is an avid half-marathon and marathon runner.
John Porter is a designer at Microsoft and Ph.D. candidate in the department of Human-Centered Design & Engineering at the University of Washington. His work focuses on the intersection of user experience, HCI, inclusive design, and accessibility.