Information assimilation in Multimedia surveillance systems

Abstract

Most multimedia surveillance systems nowadays utilize multiple types of sensors to detect events of interest as and when they occur in the environment. However, due to the asynchrony among and diversity of sensors, information assimilation, i.e. how to combine the information obtained from asynchronous and multifarious sources, is an important and challenging research problem. This thesis proposes a framework for information assimilation that addresses the issues a?? a??whena??, a??whata?? and a??howa?? to assimilate the information obtained from different media sources in order to detect events in multimedia surveillance systems. The proposed framework adopts a hierarchical probabilistic assimilation approach to detect atomic and compound events. To detect an event, our framework uses not only the media streams available at the current instant but it also utilizes their two important properties - first, accumulated past history of whether they have been providing concurring or contradictory evidences, and - second, the system designer's confidence in them. The framework also uses a dynamic programming based method to find the optimal subset of media streams based on three different criteria by having a tradeoff among the probability of the occurrence of event, the specified cost and the specified confidence. The experimental results show the utility of the proposed framework.