A novel approach to classification of facial expressions from 3D-mesh datasets using modified PCA

Abstract

We propose a novel approach to human facial expression recognition using only the shape information at a finite set of fiducial points, extracted from the 3D neutral and expressive faces. In the course of applying the technique to the facial database, BU-3DFE, which contains facial shape and 2D color ("texture") information, we extract from the images of neutral and expressive faces, salient contours in the facial interest-regions around the eyebrows, eyes, nose and mouth by invoking an active contour algorithm. The contours are then uniformly sampled and mapped onto the 3D-mesh dataset in order to generate a shape (and color) description of the interest-regions. By a matrix-algebraic operation on the shape of the neutral and expressive faces, a shape feature-matrix is computed for each expression and for each person, which is then subjected to the proposed modified PCA approach to recognize expressions. Classification results are presented to demonstrate the effectiveness of the proposed approach. It is also found that accuracy estimates compare favorably with those in the literature on facial expression recognition from 3D-mesh datasets. © 2009 Elsevier B.V. All rights reserved.