Complex query learning in semantic video search

Abstract

To address the complex query learning problem in semantic video search, this thesis proposes a three-step approach to semantic video search: concept detection, automatic semantic video search, and interactive semantic video search. In concept detection, our method proposes a higher-level semantic descriptor named ``concept bundles", which integrates multiple primitive concepts as well as the relationship between the concepts to model the visual representation of the complex semantics. In automatic semantic video search, we propose an optimal concept selection strategy to map a query to related primitive concepts and concept bundles by considering their classifier performance and semantic relatedness with respect to the query. In interactive semantic video search, to overcome the sparse relevant sample problem for complex queries, we propose to utilize a third class of video samples named ``related samples", in parallel with relevant and irrelevant samples. By mining the visual and temporal relationship between related and relevant samples, our algorithm could accelerate performance improvement of the interactive video search.