A statistical framework for image-based relighting

Abstract

With image-based relighting (IBL), one can render realistic relit images of a scene without prior knowledge of object geometry in the scene. However, traditional IBL methods require a large number of basis images, each corresponding to a lighting pattern, to estimate the surface reflectance function (SRF) of the scene. In this paper, we present a statistical approach to estimating the SRF which requires fewer basis images. We formulate the SRF estimation problem in a signal reconstruction framework. We use the principal component analysis (PCA, [1]) to show that the most effective lighting patterns for the data acquisition process are the eigenvectors of the covariance matrix of the SRFs, corresponding to the largest eigenvalues. In addition, we show that for typical SRFs, especially when the objects have Lambertian surfaces, DCT-based lighting patterns perform as well as the optimal PCA-based lighting patterns. We compare SRF estimation performance of the statistical approach with traditional IBL techniques. Experimental results show that the statistical approach can achieve better performance with fewer basis images.