Variational Approximation for Complex Regression Models

Abstract

The trend towards collecting large datasets has resulted in the need for more flexible models and fast computational approximations. My thesis reflects these themes by considering some very flexible regression models and developing fast variational approximation methods for fitting them under a Bayesian framework. Models considered include mixtures of heteroscedastic regression models, mixtures of linear mixed models and generalized linear mixed models. The advantages of variational methods as compared to MCMC methods are illustrated in model fitting, model selection and model criticism. In addition, we show that the use of reparametrization techniques such as hierarchical centering and partially noncentered parametrizations, which have been used to accelerate MCMC and EM algorithms for hierarchical models, can lead to improved convergence in variational algorithms as well. Finally, we demonstrate how stochastic approximation can be combined with variational methods to improve the accuracy of posterior approximations and make variational inference a viable approach for large datasets.