Registration using projective reconstruction for augmented reality systems

Abstract

In AR systems, registration is one of the most difficult problems currently limiting its applications. In this paper, we proposed a simple registration method using projective reconstruction. This method includes two steps: embedding and tracking. Embedding involves specifying four points to build the world coordinate system on which a virtual object will be superimposed. In tracking, a projective reconstruction technique in computer vision is used to track these specified four points to compute the modelview transformation for augmentation. This method is simple as only four points need to be specified at the embedding stage, and the virtual object can then be easily augmented onto a real video sequence. In addition, it can be extended to a general scenario using a generic projective matrix. The proposed method has three advantages: (1) It is fast because the linear least square method can be used to estimate the related matrix in the algorithm and it is not necessary to calculate the fundamental matrix in the extended case; (2) A virtual object can still be superimposed on a related area even if some parts of the specified area are occluded during the whole process; (3) This method is robust because it remains effective even when not all the reference points are detected during the whole process (in the rendering process), if at least six pairs of related reference point correspondences can be found. Several projective matrices obtained from the authors' previous work, which is unrelated with the present AR system, have been tested on this extended registration method. Experiments showed that these projective matrices can also be utilized for tracking the specified points.