Affine invariant interacting Langevin dynamics for Bayesian inference
- We propose a computational method (with acronym ALDI) for sampling from a given target distribution based on first-order (overdamped) Langevin dynamics which satisfies the property of affine invariance. The central idea of ALDI is to run an ensemble of particles with their empirical covariance serving as a preconditioner for their underlying Langevin dynamics. ALDI does not require taking the inverse or square root of the empirical covariance matrix, which enables application to high-dimensional sampling problems. The theoretical properties of ALDI are studied in terms of nondegeneracy and ergodicity. Furthermore, we study its connections to diffusion on Riemannian manifolds and Wasserstein gradient flows. Bayesian inference serves as a main application area for ALDI. In case of a forward problem with additive Gaussian measurement errors, ALDI allows for a gradient-free approximation in the spirit of the ensemble Kalman filter. A computational comparison between gradient-free and gradient-based ALDI is provided for a PDE constrainedWe propose a computational method (with acronym ALDI) for sampling from a given target distribution based on first-order (overdamped) Langevin dynamics which satisfies the property of affine invariance. The central idea of ALDI is to run an ensemble of particles with their empirical covariance serving as a preconditioner for their underlying Langevin dynamics. ALDI does not require taking the inverse or square root of the empirical covariance matrix, which enables application to high-dimensional sampling problems. The theoretical properties of ALDI are studied in terms of nondegeneracy and ergodicity. Furthermore, we study its connections to diffusion on Riemannian manifolds and Wasserstein gradient flows. Bayesian inference serves as a main application area for ALDI. In case of a forward problem with additive Gaussian measurement errors, ALDI allows for a gradient-free approximation in the spirit of the ensemble Kalman filter. A computational comparison between gradient-free and gradient-based ALDI is provided for a PDE constrained Bayesian inverse problem.…
Author details: | Alfredo Garbuno-InigoORCiD, Nikolas NüskenORCiD, Sebastian ReichORCiDGND |
---|---|
DOI: | https://doi.org/10.1137/19M1304891 |
ISSN: | 1536-0040 |
Title of parent work (English): | SIAM journal on applied dynamical systems |
Publisher: | Society for Industrial and Applied Mathematics |
Place of publishing: | Philadelphia |
Publication type: | Article |
Language: | English |
Date of first publication: | 2020/07/16 |
Publication year: | 2020 |
Release date: | 2022/10/05 |
Tag: | Bayesian inference; Langevin dynamics; affine invariance; gradient flow; gradient-free; interacting particle systems; multiplicative noise |
Volume: | 19 |
Issue: | 3 |
Number of pages: | 26 |
First page: | 1633 |
Last Page: | 1658 |
Funding institution: | Deutsche Forschungsgemeinschaft (DFG, German Science Foundation)German; Research Foundation (DFG) [SFB 1294/1 318763901, SFB 1114/2 235221301]; Paul G. Allen Family Foundation; National Science FoundationNational; Science Foundation (NSF) [AGS-1835860] |
Organizational units: | Mathematisch-Naturwissenschaftliche Fakultät / Institut für Mathematik |
DDC classification: | 5 Naturwissenschaften und Mathematik / 51 Mathematik / 510 Mathematik |
Peer review: | Referiert |
Publishing method: | Open Access / Hybrid Open-Access |
License (German): | CC-BY - Namensnennung 4.0 International |