Please use this identifier to cite or link to this item:
Title: Sampling with trusthworthy constraints: A variational gradient framework
Authors: Liu, Xingchao
Liu, Qiang
Keywords: Trustworthy Machine Learning
Bayesian Inference
Issue Date: 6-Dec-2021
Publisher: Neural Information Processing Systems
Citation: Liu, Xingchao, TONG XIN, Liu, Qiang (2021-12-06). Sampling with trusthworthy constraints: A variational gradient framework. NeurIPS 2021 Thirty-fifth Conference on Neural Information Processing Systems. ScholarBank@NUS Repository.
Abstract: Sampling-based inference and learning techniques, especially Bayesian inference, provide an essential approach to handling uncertainty in machine learning (ML). As these techniques are increasingly used in daily life, it becomes essential to safeguard the ML systems with various trustworthy-related constraints, such as fairness, safety, interpretability. Mathematically, enforcing these constraints in probabilistic inference can be cast into sampling from intractable distributions subject to general nonlinear constraints, for which practical efficient algorithms are still largely missing. In this work, we propose a family of constrained sampling algorithms which generalize Langevin Dynamics (LD) and Stein Variational Gradient Descent (SVGD) to incorporate a moment constraint specified by a general nonlinear function. By exploiting the gradient flow structure of LD and SVGD, we derive two types of algorithms for handling constraints, including a primal-dual gradient approach and the constraint controlled gradient descent approach. We investigate the continuous-time mean-field limit of these algorithms and show that they have O(1/t) convergence under mild conditions. Moreover, the LD variant converges linearly assuming that a log Sobolev like inequality holds. Various numerical experiments are conducted to demonstrate the efficiency of our algorithms in trustworthy settings.
Source Title: NeurIPS 2021 Thirty-fifth Conference on Neural Information Processing Systems
Appears in Collections:Staff Publications

Show full item record
Files in This Item:
File Description SizeFormatAccess SettingsVersion 
NeurIPS-2021-sampling-with-trusthworthy-constraints-a-variational-gradient-framework-Paper.pdf1.24 MBAdobe PDF



Page view(s)

checked on Feb 2, 2023


checked on Feb 2, 2023

Google ScholarTM


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.