Near-lossless Video Summarization

The daunting yet increasing volume of videos on the Internet brings the challenges of storage and indexing to existing online video services. Current techniques like video compression and summarization are still struggling to achieve the two often conflicting goals of low storage and high visual and semantic fidelity. In this work, we develop a new system for video summarization, called“Near-Lossless Video Summarization”(NLVS), which is able to summarize a video stream with the least information loss by using an extremely small piece of metadata. The summary consists of a set of synthesized mosaics and representative keyframes, a compressed audio stream, as well as the metadata about video structure and motion. Although at a very low compression ratio (i.e., 1/30 of H.264 baseline in average, where traditional compression techniques like H.264 fail to preserve the fidelity), the summary still can be used to reconstruct the original video (with the same duration) nearly without semantic information loss. We show that NLVS is a powerful tool for significantly reducing video storage through both objective and subjective comparisons with state-of-the-art video compression and summarization techniques.