Joint Design of Data Analysis Algorithms and User Interfaces for Video Applications

The graphical modeling paradigm provides a way of representing data through hidden causes of variability which can be estimated from the data in an unsupervised manner. Recently a lot of research has been dedicated to finding efficient inference and learning engines for graphical models in general, as well as to finding various ways of using graphical models to perform recognition, classification, segmentation, and tracking tasks in video applications. Little research, however, has focused on another advantage of a graphical model – by discovering the structural elements in the data, it renders the data much easier to browse, manipulate, or interact with. In this paper, we present several ideas on how the user interface and the data analysis tools can be designed jointly starting from an appropriate data representation scheme and a generative model based on it. We base our approach on three basic principles: (1) compatibility of the graphical model’s structure with our own perception of the world, (2) simplicity in representation, leading to more efficient inference, and (3) providing intuitive interactivity on the level of hidden causes of variability.