TofCut: Towards Robust Real-time Foreground Extraction using Time-of-flight Camera

Foreground extraction for live video sequence is a challenging task in vision. Traditional methods often make various assumptions to simplify the problem, which make them less robust in real-world applications. Recently, Time-ofFlight (TOF) cameras provide a convenient way to sense the scene depth at video frame-rate. Compared to the appearance or motion cues, depth information is less sensitive to environment changes. Motivated by the fact that TOF cameras have not been widely used in video segmentation application, we in this paper investigate the problem of performing robust, real-time bi-layer segmentation using a TOF camera and propose an effective algorithm named TofCut. TofCut combines color and depth cues in a unified probabilistic fusion framework and a novel adaptive weighting scheme is employed to control the influence of these two cues intelligently. By comparing our segmentation results with ground truth data, we demonstrate the effectiveness of TofCut on an extensive set of experimental results.