By Rob Knies, Managing Editor, Microsoft Research
Life, a great man once said, is what happens while you’re busy making other plans. And sometimes, it appears, so is art.
Three members of Microsoft Research can attest to that. The three, based on different continents and each pursuing accomplishments in widely divergent fields, found themselves improbably grouped recently in an exhibit in New York’s Museum of Modern Art (MoMA), organized by Paola Antonelli, senior curator in the museum’s Department of Architecture and Design.
Spotlight: On-demand video
AI Explainer: Foundation models and the next era of AI
Explore how the transformer architecture, larger models and more data, and in-context learning have helped advance AI from perception to creation.
Design and the Elastic Mind (opens in new tab), which opened at MoMA Feb. 24 and runs through May 12, focuses on the latest advancements in design. In the foreword of the catalog for the exhibit, Glenn D. Lowry, the museum’s director, states that the collection “comprises more than two hundred design objects and concepts that marry the most advanced scientific research with the most attentive consideration of human limitations, habits, and aspirations.”
Gavin Jancke at the Museum of Modern Art in front of his multicolored high-capacity barcode.
For Gavin Jancke (opens in new tab), director of Engineering for Microsoft Research Redmond (opens in new tab), the invitation to have his High Capacity Color Barcode (opens in new tab) featured in what is considered the world’s foremost museum of modern art came as a bit of a surprise.
“Paola Antonelli came to me,” he laughs. “Out of the blue, she sent an e-mail saying, ‘We’re really interested in exhibiting your barcode in the show.’ And I said, ‘Great!’ ”
For Indrani Medhi (opens in new tab) and Alex Taylor (opens in new tab), though, the opportunity to have their research featured in such a prestigious, unlikely setting seemed like a natural—though no less startling—extension of the nature of their work.
Indrani Medhi brings the power of computing to illiterate, first-time users in India.
“It really came as a very pleasant surprise,” says Medhi, an associate researcher in the Technology for Emerging Markets (opens in new tab) group at Microsoft Research India (opens in new tab) whose work in text-free user interfaces is being featured by MoMA. “From the beginning, we did not think of this as an art project. We just thought of it as solving a particular problem.
“But for an audience in the developed world to see a piece that combines design technology and problems of the developing world, I’m hoping it will create awareness. That’s what I want.”
For Taylor, a researcher for the Socio-Digital Systems group within Microsoft Research Cambridge (opens in new tab) whose work with vendor colleague Anab Jain resulted in two projects that are featured in the exhibit, being represented in such a setting simply furthers one of his professional goals.
“One of my major themes of research is trying to take design seriously as a contributor to research,” he says. “The MoMA exhibit is a nice example of how design can be used to present concepts to the public and get them to start engaging in questions about the possible futures of technology.”
A brief look at the Microsoft Research contributions to Design and the Elastic Mind:
Microsoft High Capacity Color Barcode (opens in new tab)
Capabilities of the High Capacity Color Barcode.
Creator: Gavin Jancke.
Artistic Background: “I’m no artist. I have trouble drawing stick figures for my kid. For me to be in an art show was very strange.”
Exhibit Description: The efficiency of information mapping and tagging technology relies on the increasing capacity and decreasing size of computer chips and other data reservoirs. The Microsoft High Capacity Color Barcode is a new system capable of holding much more retrievable information than current UPC codes. The new barcode will be useful not only to vendors but also to consumers, who will be able to scan them to obtain such information as product ratings, promotions, and pricing.
Installation: Jancke was the only one of the Microsoft Research principals featured in the show who was able to attend the exhibit opening. He described his contribution as “a three-by-two-foot mounted display on the wall, surrounded by all these other weird and wonderful things. It was great to see it there with everything else. It was one of those life experiences that I wanted to attend.”
Inclusion Means: “Having seen the other exhibits at the show, it fits with the quirky way that everything else fits in there. It’s a mixture of both working prototypes and fantastical prototypes that will never see the light of day. I can see where my colors and triangles and thinking on that fits in. It’s a sense of achievement in some way. But it’s not like I set out to create a piece of art. It’s a technology that somehow jumped the gap.”
Text-Free User Interfaces (opens in new tab)
Some of the images Indrani Medhi uses to enable illiterate, first-time users to use a PC.
Creators: Indrani Medhi and Kentaro Toyama.
Artistic Background: Medhi has a master’s degree in design from the Institute of Design at the Illinois Institute of Technology and a bachelor’s degree in architecture from the Visvesvaraya National Institute of Technology, in Nagpur, India. In July, she will begin pursuing her Ph.D. in interaction design at the Industrial Design Center of the Indian Institute of Technology, Bombay.
Exhibit Description: For the close to nine hundred million people worldwide who are illiterate, the Text-Free User Interface puts the power of computing within reach. The design is based on guidelines for computer-human interfaces “that would allow any non-literate person, on first contact with a PC, to immediately realize useful interaction with minimal or no assistance,” the designers explain. “We arrived at the design principles through an ethnographic design process. Extensive use of hand-drawn, semi-abstracted cartoons with voice annotation, aggressive mouse-over functionality, a consistent help feature, and looping full-context video dramatizing the purpose and mechanism of the application” were found to be the most effective elements for the interface. Three initial applications were created: a job-search function for the informal labor market; an interface for health-information dissemination; and an electronic map. Rigorous user evaluations show that first-time, non-literate users are able to navigate through the Text-Free User Interfaces meaningfully.
Installation: Online only.
Inclusion Means: “It feels good that technology and development are getting featured. But it was very unexpected. I feel it can motivate other people to do such things. More people who may not know about this field get to know about it.”
Objects Incognito: RFID and Body Readers (opens in new tab)
A scenario in which RFID tagging has become commonplace and people adorn themselves with beautiful, accessory-like tags and readers to sense the ever-changing social and spatial digital environment around them.
Creators: Anab Jain and Alex Taylor.
Artistic Background: Taylor has an intensive interest in design and its contributions to technology. Jain is a graduate of the Royal College of Art.
Exhibit Description: Jain and Taylor imagine a future in which radio-frequency identification (RFID) tagging will be found everywhere—objects, plants, animals, people, organs—and in which people will carry around sensors to explore the RFID-enriched landscape. In this fictional project, people will be able to manipulate and redesign the RFIDs at will, both with actual, augmented information that will add a new layer to reality, and with narrative, fantastic ones.
Installation: Online only. Jain attended the exhibit opening, but the project, in its early stages, was not yet sufficiently developed to be included in physical form.
Domestic Gubbins (opens in new tab)
The Domestic Gubbins are four fictional objects with certain abilities, designed to prompt reflections on everyday ideas of intelligence. The Gubbins were built as props and situated in an everyday context to show how they might work. People thus can imagine how it might be to live with them and what intelligence might mean in an everyday domestic context.
Creators: Anab Jain and Alex Taylor.
Exhibit Description: “The Domestic Gubbins are four fictional objects, designed to provoke people to reflect on what intelligence in machines might be like in an everyday, mundane domestic context,” Jain explains. “There are four Gubbins—Mimi, Bee, Pobel, and Snip—and each of them has a specific function. Mimi takes pictures of mundane activities and makes her own comments and stories around these activities. Bee, a split personality, uses his two parts to measure sound, temperature, and such things in the home, and then interprets these measurements in his own way. Gubbin Snip has a unique ability to sniff radio-wave emissions in the environment and associate them with strangers in the vicinity. And finally, Gubbin Pobel can sing to dying plants and bring them back to life.”
Installation: Online only.
Inclusion Means: “I’m really excited about it,” Taylor says, “because the work is still in its infancy. This gives us a good position to start with. One of the areas that we’re exploring in the group I’m in, Socio-Digital Systems, is the idea that we might extend our presence and our material body into the world through technology. Although, in some sense, this feels and sounds like science fiction, in many ways we’re already doing it. The project tries to explore those things and get people to think about what it might be like to live with RFID in different ways.”
Graphics and multimedia 
Hardware and devices 
Social sciences 
Technology for emerging markets