A curve fitting problem and its application in modeling objects in monocular image sequences

Abstract

In this paper, we present a solution to a particular curve (surface) fitting problem and demonstrate its application in modeling objects from monocular image sequences. The curve fitting algorithm is based on a modified nonparametric regression method, which forms the core contribution of this work. This method is far more effective compared to standard estimation techniques, such as the maximum likelihood estimation method, and can take into account the discontinuities present in the curve. Next, the theoretical results of this 1D curve estimation technique are extended significantly for an object modeling problem. Here, the input to the algorithm is a monocular image sequence of an object undergoing rigid motion. By using the affine camera projection geometry and a given choice of an image frame pair in the sequence, we adopt the KvD [12] model to express the depth at each point on the object as a function of the unknown out-of-plane rotation, and some measurable quantities computed directly from the optical flow. This is repeated for multiple image pairs (keeping one fixed image frame which we formally call the base image and choosing another frame from the sequence). The depth map is next estimated from these equations using the modified nonparametric regression analysis. We conducted experiments on various image sequences to verify the effectiveness of the technique. The results obtained using our curve fitting technique can be refined further by hierarchical techniques, as well as by nonlinear optimization techniques in structure from motion.