Improving subpixel stereo matching with segment evolution

Abstract

Segmentation-based approach has shown significant success in stereo matching. By assuming pixels within one image segment belong to the same 3D surface, robust depth estimation can be achieved by taking the whole segment into consideration. However, segmentation has been mostly used for stereo matching at integer disparities rather than subpixel disparities. One major reason is that small segments may be insufficient for estimating surfaces like slanted planes, while large segments may contain segmentation errors impacting the accuracy of depth estimation. In this work, we propose a segmentation-based scheme for subpixel stereo matching. Instead of using a fixed segmentation, segments are evolved to find a better support for reliable surface estimation. Given an initial estimation of segmentation and depth, the proposed algorithm jointly optimizes the segmentation and depth by evolving the segmentation at the pixel level and updating the plane parameters at the segment level. Justified with experiments performed on the Middlebury benchmark, we show that the proposed method achieves significant improvements for subpixel stereo matching.