Three ways of implementing the EM algorithm when parameters are not identifiable

Abstract

We consider situations where the incomplete nature of the observed data causes identifiability problem. Rather than imposing identifiability constraints on the parameters and then implement the EM algorithm subject to these constraints, we argue that for certain problems, an easier option is to ignore the constraints during the M-steps of the EM procedure. We also suggest a way of carrying out constrained maximization approximately by using COX and WERMUTH'S (1990) method for approximating the constrained maximizers from the unconstrained ones at each M-step. The simplicity and validity of the unconstrained EM procedure are demonstrated using three examples involving bivariate probit regression, multivariate normal order statistics model and the multinominal distribution. Potential applications to more complicated models are also outlined.