HAMILTONIAN MONTE CARLO VARIANTS

Abstract

We study two problems in statistical sampling with Hamiltonian Monte Carlo (HMC). First, we examine the difficulty of sampling a target distribution that is concentrated near a low-dimensional submanifold. This pathology occurs when we have low-noise observations and the statistical model is non-identifiable. Focusing on additive Gaussian noise models, we present Manifold Lifting, a constrained HMC methodology that transforms the concentrated posterior into a diffuse one using auxiliary variables. We further extend Manifold Lifting for general probabilistic models in the large-data regime. Second, we consider the sensitivity of HMC towards its tuning parameters. In particular, we propose the Reflected HMC to avoid having to tune the number of leapfrog steps. We suggest tuning its update rate parameter based on estimating chain auto-correlations. We demonstrate the efficiency of the proposed methods through a series of numerical experiments.