Efficient representation of lighting patterns for image-based relighting

Abstract

Image-based relighting (IBL) has become a popular research topic in both computer graphics and signal processing. IBL is the technique that renders images of a scene under different lighting conditions without prior knowledge of the object geometry and surface properties in the scene. Simply put, IBL collects images of the scene under all possible lighting conditions and process these images to render an image of the scene under a new lighting condition. To be practical, IBL strives to reduce the number of images that need to be captured, and most IBL algorithms do so by estimating the surface reflectance function (SRF) of the scene, which represents the response of each pixel in the scene to lighting from various directions. IBL hence becomes the problem of estimating the SRF using a number of lighting patterns to illuminate the scene. To minimize the number of lighting patterns needed, we propose to use a statistical approach, principal component analysis (PCA), and show that the most efficient lighting patterns should be the eigenvectors of the covariance matrix of the SRFs, corresponding to the largest eigenvalues. In addition, we show that discrete cosine transform (DCT)-based lighting patterns perform as well as the optimal PCA-based lighting patterns for typical SRFs, especially for scenes with Lambertian surfaces. Both the PCA-based and the DCT-based methods outperform existing IBL algorithms with fewer lighting patterns.