CHI ’09: Computing with a Human Touch

By Rob Knies, Managing Editor, Microsoft Research

Historically, Microsoft Research has had a big footprint during CHI, the annual conference on Human Factors in Computing Systems sponsored by the Association for Computing Machinery’s Special Interest Group on Computer-Human Interaction—and this year’s 27th gathering is no exception.

More than 12 percent of the papers accepted for this year’s conference—25 of the 204 to be presented to more than 2,000 attendees from 43 countries April 4-9 at the Hynes Convention Center in Boston—involve papers from Microsoft Research, the majority of them representing collaborations with others in the global community interested in human-computer interaction (HCI). Six of 73 short papers include Microsoft Research authors. Four panel discussions will feature Microsoft Research participation. The list goes on and on, begging the question: Why CHI?

Ken Hinckley and Meredith Ringel Morris should know. They’re serving as co-chairs of the CHI 2009 Technical Program.

“It’s a very broad topic,” noted Hinckley, a principal researcher for the Adaptive Systems and Interaction group at Microsoft Research Redmond. “The face of all our products is defined by the human experience.”

That human factor is what makes CHI so rewarding.

“It’s an exciting area for Microsoft Research because the topics capture people’s imagination about the future of computing,” added Morris, a researcher in the Adaptive Systems and Interaction group. “Things like interacting with computers using gestures or using only speech embody the excitement the public feels about the future of computing, and those topics are at the core of HCI.”

Underscoring that excitement, four of the papers boasting Microsoft Research authorship have been nominated for the Best of CHI 2009 award:

• Back-of-Device Interaction Allows Creating Very Small Touch Devices, by Patrick Baudisch of Microsoft Research and the Hasso Plattner Institute, and former Microsoft Research intern Gerry Chu of the Hasso Plattner Institute.
• A Comparison of Mobile Money-Transfer UIs for Non-Literate and Semi-Literate Users, by Indrani Medhi, Gautama S.N. Nagasena, and Kentaro Toyama, all of Microsoft Research India.
• User-Defined Gestures for Surface Computing, by Jacob O. Wobbrock of the University of Washington, Morris, and Andrew Wilson, also of Microsoft Research.
• CoSense: Enhancing Sensemaking for Collaborative Web Search, by Sharoda A. Paul of Pennsylvania State University, another former Microsoft Research intern, and Morris.

In the days leading to the opening of this year’s conference, co-sponsored by Microsoft, Hinckley and Morris identified several themes expected to galvanize the attention of attendees.

Sustainability and Green Computing

Given current interest in issues relating to ecology and the environment, it should come as little surprise that the green movement is making itself felt during CHI 2009.

“How do we actually make computing green and make it so that we’re not pumping out toxic materials or burning more coal than we need?” Hinckley said. “Understanding those problems in general and how they impact people is something that people are getting excited about.”

One such effort is outlined in It’s Not Easy Being Green: Understanding Home Computer Power Management, written by Marshini Chetty of the Georgia Institute of Technology and A.J. Bernheim Brush, Brian R. Meyers, and Paul Johns of Microsoft Research Redmond.

“They’re interested in understanding how computer technology can be more power-efficient,” Morris said, “and how it can do that in a way that fits better into a user’s lifestyle and existing practices.”

Technologies for Developing Countries

Microsoft Research’s global presence enables it to investigate the potential for innovation in a variety of social settings.

“Microsoft Research’s lab in Bangalore gives us a great opportunity as a company to explore technologies for developing countries,” Morris said. “Kentaro Toyama’s group has been doing some interesting work on user interfaces for people who can’t read.”

That paper, the best-paper nominee written by Medhi, Nagasena, and Toyama, is a product of the latter’s Technology for Emerging Markets team, as is Comparing Semiliterate and Illiterate Users’ Ability to Transition from Audio+Text to Text-Only Interaction, by Leah Findlater of the University of British Columbia, Ravin Balakrishnan of the University of Toronto, and Toyama.

Another emerging-markets paper of note: A Mischief of Mice: Examining Children’s Performance in Single Display Groupware Systems with 1 to 32 Mice, written by Neema Moraveji of Stanford University, Kori Inkpen and Edward Cutrell of Microsoft Research Redmond, and Balakrishnan.

“This looks at how you can support education in the developing world,” Morris said, “by connecting multiple mice to a single computer so you can cheaply have multiple students engaging with technology.

“The researchers conducted a study to learn how that technology scales with different numbers of students using the mice at the same time. They’ll be presenting some of those results.”

Web and Communication Technologies

Social computing is a particularly fertile field these days, and a number of CHI 2009 papers address various aspects of this burgeoning trend, among them mimir: A Market-Based Real-Time Question and Answer Service, written by Gary Hsieh of Carnegie Mellon University and Scott Counts, a researcher in the Visualization and Interaction for Business and Entertainment (VIBE) group at Microsoft Research Redmond.

“There are services where you have a question and you can’t find the answer with your Web search,” Morris explained, “so you want a real person to answer that question. This is about how you can ensure higher-quality answers and good matches between people who have the questions and those who have the answers, by looking at economic models you might apply.”

Her own CoSense paper, co-authored with Paul, suggests ways to enhance the experience of collective search, what she terms “collaborative sensemaking.”

“If you have groups of users engaged in collaborative Web searches, working together to find information online,” Morris said, “how can you provide a more rich user interface to support that experience, to help people move beyond just doing a search together to using the information they’ve found for some goal or task in a collaborative manner?”

Another pertinent effort is What Do You See When You’re Surfing: Using Eye Tracking to Predict Salient Regions of Web Pages, by Georg Buscher of the Deutsches Forschungszentrum für Künstliche Intelligenz, Cutrell, and Morris.

“We’re using eye tracking to understand what parts of Web pages people spend the most time looking at and understanding how to make very compact visual representations of Web pages,” Morris said. “That could be helpful to find information more quickly by having the representation include the most memorable aspects of the page.”

A collaboration between Microsoft Research and Microsoft Live Labs, Visual Snippets: Summarizing Web Pages for Search and Revisitation, by Jaime Teevan, Cutrell, and Danyel Fisher of Microsoft Research Redmond, Steven M. Drucker and Gonzalo Ramos of Live Labs, Paul André of the University of Southampton, and Chang Hu of the University of Maryland, explores visual cues to help people find Web pages they have previously viewed.

“That paper look at other techniques for creating compact representations of Web pages,” Morris said, “and evaluates how well these different representations perform for re-finding information.”

And André, Teevan, and Susan T. Dumais of Microsoft Research have contributed the intriguingly named short paper From X-Rays to Silly Putty via Uranus: Serendipity and Its Role in Web Search.

Technology’s Role in Healthcare

“Medicine is littered with failed experiments in deploying technology that goes unused,” Hinckley noted. “The CHI conference includes studies on how people use these artifacts and how you can make them fit people’s existing work practices. That’s where we can get fresh insights into those problems.”

Two members of Microsoft Research, Ryen White and Desney Tan, will be participating in a 14-researcher panel discussion entitled “Interacting with eHealth.” The panel also includes Tim Berners-Lee of the Massachusetts Institute of Technology, who is credited with the invention of the World Wide Web.

“Microsoft understands healthcare technology is an important area,” Morris said, “and we have the potential to do leading-edge research in this area and to share our research with product groups within Microsoft, such as Microsoft HealthVault™. We’re trying to learn more about what people in the community think is important.”

Pen and Touch-Computing Technologies

Microsoft Surface™ has received a lot of attention in recent months, but the concept of surface computing extends beyond the table-based domain. Even small mobile devices can incorporate direct-touch interaction.

“I see those as two ends of a continuum,” Hinckley said. “Imagine bigger surfaces that are touch-driven or operated by gestures. There are a lot of ways of interacting naturally.

“There’s a lot of interest in the community in understanding how you interact with these types of devices, how you leverage the rich multitouch capabilities. You can do things with two hands, or multiple people can do things together. How do you understand that and go to something that’s much nicer, with a much larger display space, that is much richer in terms of what you can do?”

Among the papers that discuss touch computing, pen computing, and surface-like technologies is one from Hinckley himself, Codex: A Dual Screen Tablet Computer, co-written with Morgan Dixon of the University of Washington; Raman Sarin of Microsoft Research; François Guimbretière, formerly of the University of Maryland and now at Cornell University, and Balakrishnan. The Codex paper concerns the potential for a dual-display device that has Tablet PC input and two screens.

Morris mentioned Baudisch’s Back-of-Device Interaction Allows Creating Very Small Touch Devices paper, which uses input from fingers behind a handheld device to solve the problem of screen occlusion, as another innovative technology to be discussed during CHI. She, too, has contributed work in this area.

“There’s a project that Andy Wilson, Jake Wobbrock, and I worked on [User-Defined Gestures for Surface Computing] about defining new types of gestures that you would use on interactive surfaces,” she stated. “We propose a new methodology for defining the gestures based on observations of the types of gestures that users naturally engage in on surfaces.”

Also worthy of attention are works that explore the potential of surface technologies for artistic applications such as VJing and collaborative audio editing: Turning the Tables: An Interactive Surface for VJing, by Stuart Taylor, Shahram Izadi, David S. Kirk, Richard Harper, and Armando Garcia-Mendoza, all of Microsoft Research Cambridge, and Dynamic Mapping of Physical Controls for Tabletop Groupware, written by Rebecca Fiebrink of Princeton University, Morris, and her husband, Dan Morris, also a researcher at Microsoft Research Redmond.

Usable Security and Privacy

Secure computing is a necessity in the Internet era, but that doesn’t mean that it’s always convenient.

“My whole interaction with a mobile device might be for five or 10 seconds, if I just want to jot a quick note in my notebook,” Hinckley said. “If I have to type in a password to do that, it basically makes it prohibitive to use the device, because it’s so frustrating to type a password on small-screen devices.

“In terms of usability, having security mechanisms that can deal with those tight timing constraints is important.”

A CHI paper that addresses similar concerns is It’s Not What You Know, but Who You Know: A Social Approach to Last-Resort Authentication, by Stuart Schechter of Microsoft Research Redmond, Serge Egelman of Carnegie Mellon University, and Robert W. Reeder, a user-experience researcher in Microsoft’s Trustworthy Computing group.

Of course, the CHI focus isn’t limited to these six themes. HCI offers wide latitude for exploration.

“Everyone is seeing the netbook devices hitting,” Hinckley observed. “It’s one of the few sectors in the entire economy that’s growing. What happens when the price of these keeps getting driven down and they have fewer capabilities at lower cost? Suddenly, touch input makes a lot of sense.”

There’ll be plenty more going on during CHI 2009. Jonathan Grudin, a principal researcher at Microsoft Research Redmond, is scheduled to serve as instructor for a course entitled “HCI History: Trajectories into the Future.” Brush will lead a special-interest group called “Designing for Families.” Toyama assisted Jay Chen and Lakshminarayanan Subramanian of New York University on Web Search and Browsing Behavior Under Poor Connectivity, part of the Student Research Competition. And Microsoft Research Cambridge’s Alex S. Taylor and Harper, along with Laurel Swan of the Royal College of Art, will present their Transactions on Computer-Human Interaction paper Making Place for Clutter and Other Ideas at Home.

Two video showcases feature Microsoft Research personnel: CHIstory, by Grudin, and Digital Heritage, by Aditya Sankar, Archana Prasad, Joseph Joy, Naren Datha, and Ajay Manchepalli of Microsoft Research India. Toyama co-organized an opening-weekend workshop called Human-Centered Computing in International Development, and Dan Morris likewise co-organized a workshop on Computational Creativity Support: Using Algorithms and Machine Learning to Help People Be More Creative.

And four panel discussions will include Microsoft Research contributions:

• The Beauty Dilemma, including Bill Buxton, principal researcher, and Mary Czerwinski, research area manager for the VIBE group.
• Scientometric Analysis of the CHI Proceedings, including danah boyd, researcher at Microsoft Research New England.
• Growing Up Programming: Democratizing the Creation of Dynamic Interactive Media, including Matthew MacLaurin, principal program manager for the Creative Systems group at Microsoft Research Redmond.
• Interacting with eHealth: Toward Grand Challenges for HCI, including White and Tan.

In addition to Technical Program co-chairs Hinckley and Morris, Bongshin Lee and Nathalie Riche of the VIBE group served as the conference’s proceedings chairs.

Such efforts can prove rewarding in themselves.

“It’s a great way to learn about all the latest work that’s going on in the field of HCI,” Morris said, “and make sure that we’re aware of the current trends.”

And, Hinckley added, getting people involved and making contacts can pay dividends over the course of a career.

“It’s also a great way to shape what kinds of research are encouraged by the conference,” he said, “trying to make the review process open to a broad range of contributions.

“Making sure that the best people we know throughout the CHI community and at universities and other research labs get involved, and forging those relationships, as well, with Microsoft—they’re all doors we can go knock on in the future.”

Opens in a new tab