Multimodal approach to human-face detection and tracking

Abstract

The constructive need for robots to coexist with humans requires human-machine interaction. It is a challenge to operate these robots in such dynamic environments, which requires continuous decision-making and environment-attribute update in real-time. An autonomous robot guide is well suitable in places such as museums, libraries, schools, hospital, etc. This paper addresses a scenario where a robot tracks and follows a human. A neural network is utilized to learn the skin and nonskin colors. The skin-color probability map is utilized for skin classification and morphology-based preprocessing. Heuristic rule is used for face-ratio analysis and Bayesian cost analysis for label classification. A face-detection module, based on a 2-D color model in the YCrCb and YUV color space, is selected over the traditional skin-color model in a 3-D color space. A modified Continuously Adaptive Mean Shift tracking mechanism in a 1-D Hue, Saturation, and Value color space is developed and implemented onto the mobile robot. In addition to the visual cues, the tracking process considers 16 sonar scan and tactile sensor readings from the robot to generate a robust measure of the person's distance from the robot. The robot thus decides an appropriate action, namely, to follow the human subject and perform obstacle avoidance. The proposed approach is orientation invariant under varying lighting conditions and invariant to natural transformations such as translation, rotation, and scaling. Such a multimodal solution is effective for face detection and tracking. © 2008 IEEE.