Efficient derivative-free Bayesian inference for large-scale inverse problems
- We consider Bayesian inference for large-scale inverse problems, where computational challenges arise from the need for repeated evaluations of an expensive forward model.
This renders most Markov chain Monte Carlo approaches infeasible, since they typically require O(10(4)) model runs, or more.
Moreover, the forward model is often given as a black box or is impractical to differentiate.
Therefore derivative-free algorithms are highly desirable. We propose a framework, which is built on Kalman methodology, to efficiently perform Bayesian inference in such inverse problems.
The basic method is based on an approximation of the filtering distribution of a novel mean-field dynamical system, into which the inverse problem is embedded as an observation operator.
Theoretical properties are established for linear inverse problems, demonstrating that the desired Bayesian posterior is given by the steady state of the law of the filtering distribution of the mean-field dynamical system, and proving exponential convergence to it.
ThisWe consider Bayesian inference for large-scale inverse problems, where computational challenges arise from the need for repeated evaluations of an expensive forward model.
This renders most Markov chain Monte Carlo approaches infeasible, since they typically require O(10(4)) model runs, or more.
Moreover, the forward model is often given as a black box or is impractical to differentiate.
Therefore derivative-free algorithms are highly desirable. We propose a framework, which is built on Kalman methodology, to efficiently perform Bayesian inference in such inverse problems.
The basic method is based on an approximation of the filtering distribution of a novel mean-field dynamical system, into which the inverse problem is embedded as an observation operator.
Theoretical properties are established for linear inverse problems, demonstrating that the desired Bayesian posterior is given by the steady state of the law of the filtering distribution of the mean-field dynamical system, and proving exponential convergence to it.
This suggests that, for nonlinear problems which are close to Gaussian, sequentially computing this law provides the basis for efficient iterative methods to approximate the Bayesian posterior.
Ensemble methods are applied to obtain interacting particle system approximations of the filtering distribution of the mean-field model; and practical strategies to further reduce the computational and memory cost of the methodology are presented, including low-rank approximation and a bi-fidelity approach.
The effectiveness of the framework is demonstrated in several numerical experiments, including proof-of-concept linear/nonlinear examples and two large-scale applications: learning of permeability parameters in subsurface flow; and learning subgrid-scale parameters in a global climate model.
Moreover, the stochastic ensemble Kalman filter and various ensemble square-root Kalman filters are all employed and are compared numerically.
The results demonstrate that the proposed method, based on exponential convergence to the filtering distribution of a mean-field dynamical system, is competitive with pre-existing Kalman-based methods for inverse problems.…
Author details: | Daniel Zhengyu Huang, Jiaoyang Huang, Sebastian ReichORCiDGND, Andrew M. Stuart |
---|---|
DOI: | https://doi.org/10.1088/1361-6420/ac99fa |
ISSN: | 0266-5611 |
ISSN: | 1361-6420 |
Title of parent work (English): | Inverse problems : an international journal of inverse problems, inverse methods and computerised inversion of data |
Publisher: | IOP Publ. Ltd. |
Place of publishing: | Bristol |
Publication type: | Article |
Language: | English |
Date of first publication: | 2022/10/31 |
Publication year: | 2022 |
Release date: | 2024/08/16 |
Tag: | Bayesian inference; derivative-free optimization; ensemble Kalman filter; interacting particle system; inverse problem; mean-field dynamical system; uncertainty quantification |
Volume: | 38 |
Issue: | 12 |
Article number: | 125006 |
Number of pages: | 40 |
Funding institution: | Office of Naval Research (ONR) [N00014-17-1-2079]; Simons Foundation;; Deutsche Forschungsgemeinschaft (DFG) [318763901-SFB1294] |
Organizational units: | Mathematisch-Naturwissenschaftliche Fakultät / Institut für Mathematik |
DDC classification: | 5 Naturwissenschaften und Mathematik / 51 Mathematik / 510 Mathematik |
Peer review: | Referiert |
License (German): | ![]() |