Lookout System: National Television Commercial (1998)

The Lookout project was an early exploration of the promise of machine learning, mixed-initiative interaction, and multimodal interaction as a foundation for intelligent services. In this widely broadcast, fast-paced 1998 television commercial, project lead Eric Horvitz demonstrates several capabilities of the system, including Lookout’s ability to recognize intentions, to decide when to perform dialog, to identify the best time to intervene, and to take actions to help with daily activities. The system brought together machine learning, decision making under uncertainty, speech recognition, natural language processing, dialog, and use of animations to relay gestures, such as the communication of system confusion. Lookout learned to recognize the intentions of a user (e.g., to schedule an appointment based on the message at focus of attention) via “streaming supervision” of end-users’ activities. Confidence in speech recognition was used to identify out-of-vocabulary speech for robustness in the open world. The television commercial shares Eric’s aspirations of the late 1990s for where intelligent systems and services might go one day in supporting and empowering people in their daily lives. More on Lookout.