Juggling the Effects of Latency: Motion Prediction Approaches to Reducing Latency in Dynamic Projector-Camera Systems

Projector-camera (pro-cam) systems afford a wide range of interactive possibilities, combining both natural and mixed-reality 3D interaction. However, the latency inherent within these systems can cause the projection to ‘slip’ from its intended target, detracting from the overall experience. Because of this, pro-cam systems have typically shied away from truly dynamic scenarios. In turn, research has been exploring latency reduction techniques across a range of domains, but these techniques typically focus on custom hardware, limiting their widespread adoption. We explore software-only predictive approaches to minimize the effects of latency in pro-cam systems. In this paper, we focus our predictive approaches on real-world objects under fast motion and on-body projection, improving projection accuracy on fast moving targets. Alongside this we explore automatic latency measurement techniques, allowing our system to determine and account for its own latency. We detail predictive approaches and provide results of a series of empirical investigations; achieving a 37% improvement in projection accuracy on objects in free flight (at speeds approaching 5m/s), and a 43% improvement in on body projection (with movement circa 1.5m/s). Through our work we aim to facilitate the wider exploration of pro-cam systems for 3D interaction in dynamic settings and showcase the accuracy achievable with off-the-shelf hardware.