Concept Drift Detection for Multivariate Data Streams and Temporal Segmentation of Daylong Egocentric Videos

Egocentric videos, with their lengthy and unstructured nature, necessitate temporal segmentation as a crucial preprocessing step for many high-level inference tasks, often spanning hours and characterized by gradual changes. The wearer’s head movement causing frequent and drastic scene changes complicates matters, as traditional methods like Markov Random Field (MRF) pipelines struggle with their continuous boundaries. Deep learning approaches such as Long Short Term Memory (LSTM) networks also fall short due to their limited context gathering capability. We introduce an innovative unsupervised temporal segmentation technique tailored for day-long egocentric videos. By detecting concept drift within a dynamic sequence of frames, we establish statistically bounded thresholds to identify changes in underlying distributions between adjacent segments. Our method significantly enhances the state-of-the-art f-measure for daylong egocentric video datasets and associated photostream datasets derived from them: HUJI (73.01%, 59.44%), UTEgo (58.41%, 60.61%), and Disney (67.63%, 68.83%).