By Suzanne Ross, Writer, Microsoft Research
How would you feel if a co-worker barged into your office every few minutes to blurt out updates about their life or project? You might tell them that you are busy, but the damage has been done. You’ve been interrupted, and getting back on task might be difficult.
Eric Horvitz and his team in the Adaptive Systems and Interaction group have been working for a number of years on what they call the Attentional User Interface (AUI) project. «Human attention is the scarcest resource in computing, and we’ve been pursuing the creation of systems that learn, sense, and reason about this precious commodity,» said Horvitz.
Spotlight: blog post
GraphRAG auto-tuning provides rapid adaptation to new domains
GraphRAG uses LLM-generated knowledge graphs to substantially improve complex Q&A over retrieval-augmented generation (RAG). Discover automatic tuning of GraphRAG for new datasets, making it more accurate and relevant.
The AUI project focuses on developing and testing automated learning and reasoning methods. These methods can infer how busy a user is or will be, or determine their focus of attention. As part of this work, Horvitz’s team has studied people’s ability to continue on task after being interrupted while performing tasks at different times and in different ways.
But considering the costliness of interruptions is not the complete story. Horvitz’s team has also created a number of applications that can estimate the value of the incoming message and phone calls and balance them with the cost of interruption.
One application developed by Horvitz’s team, called BusyBody, is a tool that can monitor your activity and predict how interruptable you are.
«We’re being pulled in so many directions by daily events and needs. In one sense networked computing has become an incredible tool for getting things done and staying in touch more efficiently. On the other hand, the platform supports so many communication applications that it’s also become a major switching post and center of distraction.
«We’ve studied people in our user labs to find out how they recover from distractions, and when is the best time to interrupt someone. Some interruptions are courteous, such as a tap on the shoulder or a knock on the door. We wanted to build our application to be courteous and intelligent about the nature and timing of interruptions.
«We have pursued methods that can learn to diagnose the cost of interrupting someone by considering observations about their activities and context. We consider streams of sensed events about a user’s computing activity, such as information about how users shift among or focus on different software applications. BusyBody also considers other classes of information such as the current location of the user via wireless signals, whether nearby conversation is detected, the time of day and day of week, and details about appointments on their calendar. At run time, BusyBody informs messaging and call control systems about if, when, and how to interrupt users. This balances, in an elegant way, the cost of interruption with the cost of having important information or communications delayed.»
Busybody is an evolution of heavier-weight offline training systems packaged into an easy to train, self-contained system. In a training phase, the Busybody interface pops up a busy palette every once in awhile. The palette allows users to specify how busy they are at that moment. In the background, the system is logging events about the user’s activities and context, which it maps to the user’s input. Busybody uses this case library of events to train and re-train Bayesian models that can predict a user’s state of busyness.
In related experiments, Horvitz and his team studied different combinations of observations. In one case, they discovered that if they allowed a system to «see» a user’s calendar, it would build models that didn’t make use of cameras or microphones. But if they didn’t allow the system to access the user’s calendar, it would build models that used microphones and video cameras to find out about the user’s context.
«You hide the calendar and now the system wants to look and listen at what’s happening in the user’s office. You can almost feel the sparking of intelligence there,» said Horvitz.
Beyond the use of BusyBody to guide messages and alerting, it has also been used in another research project called Coordinate. Coordinate is a collaboration tool that lets other people inspect a user’s busyness. Privacy is important, so the system allows users to define who can access this information. For example, a user may wish to share this information only with co-workers and close family.
BusyBody could allow us to be more productive by minimizing costly interruptions in our daily lives.
Artificial intelligence