A probabilistic framework for geometry and motion reconstruction using prior information

Abstract

In this paper, we propose a probabilistic framework for reconstructing scene geometry and object motion utilizing prior knowledge of a class of scenes, eg, scenes captured by a camera mounted on a vehicle driving through city streets. In this framework, we assume the video camera is calibrated, ie, the intrinsic and extrinsic parameters are known all the time. While we assume a single camera moving during the capture, the framework can be generalized to multiple stationary or moving cameras as well. Traditional approaches match the points, lines or patches in multiple images to reconstruct scene geometry and object motion. The proposed framework also takes advantage of each patch's appearance and location to infer its orientation and motion direction using prior information based on statistical learning from training data. The prior information hence enhances the performance of geometry and motion reconstruction. We show that the prior-based 3D reconstruction outperformed traditional 3D reconstruction with synthetic data and real data, especially in textureless areas for geometry estimation and faraway areas for motion estimation.