Incremental Coordination: Attention-Centric Speech Production in a Physically Situated Conversational Agent

Inspired by studies of human-human conversations, we present methods for incrementally coordinating speech production with listeners’ visual foci of attention. We introduce a model that considers the demands and availability of listeners’ attention at the onset and throughout the production of system utterances, and that incrementally coordinates speech synthesis with the listener’s gaze. We present an implementation and deployment of the model in a physically situated dialog sys-tem and discuss lessons learned.