New depth cue based algorithm for background-foreground segmentation

Abstract

In this paper, we present a method for segmenting the interesting foreground from the background using a novel depth cue based algorithm. The input to the algorithm are two pairs of images, the first being the stereo pair corresponding to the background image only (called the background pair), and the second corresponds to the stereo pair when the object(s) of interest is present in front of the background (called the composite pair). Since we use stereo images rather than monocular images, we can utilize the fact that the interesting foreground has a depth/disparity value which is different from the corresponding values for the background. Under situations such as poor lighting conditions, or when lighting conditions change continuously, it may be quite unreliable to extract the foreground by the process of subtracting the composite image from its background counterpart, followed by a thresholding process. Also, the camera noise is usually unknown, in general. Instead, we compute the disparity image corresponding to the background stereo pair, and validate the disparity values for the composite pair. A point belonging to the foreground will certainly have a higher disparity value. Based on the novel depth cue based measure introduced in this paper, it would fail the validation process and hence would be classified as a foreground pixel. The other notable point is that the computationally expensive stereo matching process is performed offline, and hence the segmentation process is quite fast.