Hello! I’m Amy, Chief User Wizard for Microsoft Soundscape. In this role I wear many hats, all focused on user success: part researcher, part experience designer, and part user advocate. In short, I am the voice of the user, gathering, prioritizing and communicating user needs to the team. While my educational journey includes a PhD in computer science, my passion has always been firmly on the human side of computing, with an overarching theme of empowering people in the face of everyday constraints. My early work, which predated the iPhone, looked at increasing the usability of mobile touchscreen devices under conditions of low visual attention and hand availability. Later projects involved improving user productivity on small screen devices in critical situations, empowering emergency room patients through real-time phone-based status updates and improving learning and compliance for physical therapy patients using tiny handheld video projectors.
As the wife of a wheelchair user, I have deep familiarity with and sensitivity to the usability gaps in technology and the world in general when viewed through the lens of environmental, situational, and personal constraints. One might even say that all my experience to date has led me to this exciting challenge of bringing a mobile awareness and empowerment system to those with low vision and blindness. To give you a sense of what this means in a day to day sense, let me walk you through the process of how the recent “Markers” feature came to life in Microsoft Soundscape.
The Markers Story
When we launched Soundscape in March 2018, my job transitioned from being largely about field work—hitting the streets with users to gather information about what was working and what wasn’t working for them about the Soundscape experience—to one of having my ear to the ground listening to feedback from users «in the wild,» real folks out there in the real world using Soundscape as part of their life experience! Another way to think about it, is that before the product launch, I was on a first-name basis with the people who were giving us feedback. While personal, in-depth sessions are excellent for usability testing and understanding nuances of use, it is not able to capture the full range of tasks people want to perform with the app, or the range of situations that may cause people trouble or frustration, or even the different levels of experience and expectations people have when using Soundscape for the first time.
It was at this time that we also launched our «Insider’s» program of users who would receive pre-release versions of Soundscape that we could get feedback on before each update to the AppStore. Once we launched in the AppStore, however, we began to receive email from across the US and the UK about the highs and lows of users’ experiences with Soundscape. While this input was necessarily less detailed than the type I gathered when looking over people’s shoulders as they used the app and from asking direct follow-up questions about their expectations and intent in the moment, the benefit of the email feedback channel was that it opened our eyes to a broad range of very specific real-world scenarios for which people were turning to Soundscape and when things were going well and when they weren’t. In these larger numbers, we could begin classifying and categorizing feedback to move us from a mostly qualitative understanding of the user experience, to being able to make decisions based on quantitative data.
Fairly quickly we began to see a theme that Soundscape users were not hearing about the places in their environment that were most relevant to them. And if you think about it, it makes sense that most data sources can’t provide the level of detail that matters from a practical perspective for someone with low vision to navigate a space. Their needs go beyond simply street names and points of interest. One might want to know, for example, if an intersection is signaled, how many lanes of traffic a crosswalk has, if there is construction on the block they are planning to walk, or even if there exists a side walk on the road they plan to turn down. There is an entire field of academic research that is attempting to help answer these questions, and the data source we use is a good starting point from that perspective because it is crowd sourced and so can include such user-centered details that traditional mapping programs do not.
Yet, we had a hunch that adding further descriptive detail to the map was missing part of the story. By asking users further questions, conducting focus groups, and going out with people one on one in their own neighborhoods, what we discovered is that the places that are important to people are often very personal, and not likely to ever be part of a public data set. We were hearing people tell us about places that they wanted to highlight in their environment because it had unique meaning based on their view on the world and context of use. For example, people wanted Soundscape to know about their house and have it referred to as «Home», not their address, and similarly for their friend’s houses.
At other times, a user’s view of the world is more granular than the one typically represented in public data sources. As an example, one user described a need to mark arbitrary locations in large housing complexes, corporate or college campuses, and parks to aid their navigation and in locating specific features of those spaces; so they marked items such as a specific unit in a housing complex, a route through a campus, or a bench in a park. At other times, people have only a short-term need for keeping track of a location; for example, as one user described, when they go for a walk on a beach, it would be helpful to keep track of the specific access path they entered the beach from so that they can easily return to it. Other environmental features are of interest to users because of the importance of that route for walking; we have had folks highlight a pesky low hanging branch, uneven pavement, their guide dog’s favorite patch of grass, or a fork in a footpath that was frequently overshot. These are not places that have a name, or an address, or would even be relevant to anyone other than themselves.
In summary, what was clear was that people needed a way to personalize the Soundscape experience to reflect their unique usage and needs; and this is how we came to introduce Markers, because they allow users to mark up their world. A typical use of markers is to «Mark your current location» and give it a personally meaningful name. Of course text entry is not always practical on the go, so we took pains to give Markers descriptive default names that would help users differentiate one Marker from another when at a later point they had time to edit the name. The Markers feature also allows people to rename existing points of interest, such as renaming a lengthy bus stop name to «Bus to Work». It is important to note that a fundamental design principle of Markers has been to preserve users’ privacy and identity even as we allow users to personalize their Soundscape experience.
Since the launch of the Markers feature, we’ve received incredibly positive feedback on it. But this is not the end of the story for this feature by any stretch. Each step forward brings new challenges, use scenarios, and opportunities to iterate and improve. For example, with the introduction of Markers, we are seeing more interest from users in surfacing the status of the GPS signal, which suggests that people are using Markers in a different way than general points of interest. We are keeping a close eye on all the feedback we receive and I will continue my in-person sessions in the field to tease apart the nuances of the use of Markers in different situations. And of course this is just one example among many others in our listen-design-build-evaluate cycle at the heart of our mission to connect our users more closely to their surroundings and empower users to move more confidently within the world.
Learning from the community
For example, another area that we have been investing in is how to help Soundscape users get up and running with the app as quickly and confidently as possible. We know that there is a significant population of people that use Soundscape several times a week and love it, while others use Soundscape more sparingly. While this is not an issue that is unique to Soundscape by any means, we also know that Soundscape is a different kind of app with several brand new concepts to become comfortable with. We also recognize that it can take a bit of a leap of faith to be willing to add another element to an activity that may already include other mobility devices in addition to a high need for environmental awareness and concentration. The in-app tutorials are a great start but we have lots of ideas about how to engage users more effectively in the first couple of days of downloading Soundscape, including experiences they can have in a familiar environment such as their home.
Another aspect of Soundscape that we have continuing discussions around is the question of customization. We get a fair amount of feedback suggesting that users want quite granular control over certain aspects of the Soundscape experience. This is a tricky area because doing so will likely make the interface more complicated, which we do not want to do. Also, there are philosophical questions around whether certain customizations are in keeping with the goals of Soundscape: that is to say, if Soundscape’s goal is to be an awareness app, is it doing its job if the user turns off certain categories of information? In an ideal world, we would instead be able to leverage 3D audio and human perception so that we can present all the information we have, but in a way that allows users to selectively and naturally tune out the information that is not relevant to them, not unlike the way that the background hum of a distant highway or the conversation of two people across the room does not interfere with a conversation you are having with a friend. But we are certainly not there yet and in the meantime we need to be attuned to what will make the app most useful to people today.
Personal reflections
If there’s one thing I’ve learned over the time I have worked on the Soundscape team, it is to never underestimate the power and reach of Soundscape. What I mean by this is that because Soundscape can be demonstrated in just a matter of minutes, first impressions of Soundscape can conclude that it is a fairly simple app, and I was no exception. But what I have heard time and again during my engagements with users is that Soundscape is nothing short of transformative in its ability to empower users and foster confidence and independence when they are out and about. The design challenges go far beyond the basics of optimizing button placement and function, but instead span questions of how to more seamlessly integrate with users’ existing mobility and wayfinding tools, how to support lightweight and even hands-free interaction, and how to ensure that the information that Soundscape provides is unobtrusive and respects the individual’s context of use.
Furthermore, Soundscape’s value isn’t limited to a particular user population or scenario, as dozens of environments, such as backwoods hiking or complicated campus layouts, or even traveling to a new place, all pose navigational puzzles that could be more easily managed with the aid of 3D audio regardless of a person’s level of vision.
On a personal note Soundscape has fundamentally changed how I look at the world. I will never experience an intersection crossing again without considering the challenges that the design poses for those with blindness or low vision and I am grateful to be part of a team of passionate individuals who are dedicated to leveraging spatial sound to make the world a more navigable and accessible place for everyone.