Feature-based detection and tracking of individuals in dense crowds

Abstract

Visual surveillance research is an important topic in computer vision and has received increasing attention ever since the nine-eleven attacks in 2001. Despite many existing works on detection, tracking and behavior recognition in different video surveillance environments, only a few have considered densely crowded places. This is an issue that needs to be addressed as crowded areas should be of great concern since terrorist attacks in such places can achieve maximum fatalities and provide cover for the perpetrators to escape unnoticed. This forms our motivation to detect and track people in dense crowds. Here, it is common for a person to be significantly occluded, where the visible part of a person in any camera view can be unpredictable, making it difficult to use regular windows, shapes or human models. Therefore, available methods which are human-specific model-based, region-based and contour-based are not suitable for reliably detecting and tracking individuals in this scenario.In this thesis, we propose a feature-based approach to detect the head of an individual, which is possibly the most unoccluded part of a person in dense crowds, and track it to facilitate further processing like identification and behavior analysis. There are no salient elements such as areas, colors and edges that can reliably represent the head of an individual in dense crowds. Therefore, we use Haar-like features to train a local head detector offline and further propose a two step post-processing procedure to improve the performance of the detector. The first step creates a color bin image from each of the initially detected windows. Every color bin image created is then classified as a correct detection or a false alarm using a trained cascade of boosted classifiers that also uses Haar-like features. The second step exploits the use of a weak perspective model for a single uncalibrated camera to further reduce the false alarm rate. This step simply relies on the 2D image size of the detected windows and their 2D locations in the image frame. However, here, we assume that the crowd is distributed over a plane and that the individuals within the crowd have the same 3D world size.We also propose a method for tracking heads in detected windows, tailored to the the scenario of dense crowds. Based on spatiotemporal measurements, our approach uses several Kanade-Lucas-Tomasi (KLT) feature points in a Bayesian framework. Here, the locations of the feature points are used to define a prior term and the motion coherency of these feature points is used to define the likelihood term. During time instants when the tracker infers a significantly occluded head, a linear approximation method is used to estimate the track. Additional characteristics of the tracker, such as robustness against scaling and rotational motions, are also proposed.Finally, we propose a method to find the best frontal facial view of the detected and tracked person from among the multiple images in a video sequence, so as to optimize the performance of further processing, such as face recognition.Results of the proposed head detector are presented in the form of Receiver Operating Characteristics (ROC) curves; for instance, at a 79.0% detection accuracy, the false alarm rate is 20.3%. Results of the proposed tracking system are presented qualitatively on densely crowded scenes and many other tracking scenarios, including vehicle tracking. Results of the proposed method in finding the best frontal facial view are presented with respect to person dependency, low pixel resolution, occlusion problems and in densely crowded scenes.