Video analysis and indexing in compressed domain

Abstract

In this thesis, we explore the compressed domain video analysis and indexing in the Automated Transaction Service (ATS) surveillance domain and the sports video domain, respectively. Since human faces provide a useful cue in indexing video data, we propose a coarse-to-fine scheme to estimate the clear degree of faces present in MPEG compressed video of Head-and-Shoulders style. Using representative key frames containing a clearer face, much surveillance video is to be attached with a novel semantic label, namely clear face. The Incremental Focus of Attention (IFA) architecture is taken to combine pixel domain processing with transform domain processing to achieve robust and efficient face pose tracking in very cluttered scenes. Aiming at the computational scheme to understand the high-level semantics of the structured video content, we have developed a unified framework for semantic shot classification in sports videos featured higher structured constraints. With correctly classified sports video shots, further structural and temporal analysis, such as event detection, video skimming, table of content, etc., will be greatly facilitated.