TY - JOUR A1 - Leung, Tsz Yan A1 - Leutbecher, Martin A1 - Reich, Sebastian A1 - Shepherd, Theodore G. T1 - Forecast verification BT - relating deterministic and probabilistic metrics JF - Quarterly journal of the Royal Meteorological Society N2 - The philosophy of forecast verification is rather different between deterministic and probabilistic verification metrics: generally speaking, deterministic metrics measure differences, whereas probabilistic metrics assess reliability and sharpness of predictive distributions. This article considers the root-mean-square error (RMSE), which can be seen as a deterministic metric, and the probabilistic metric Continuous Ranked Probability Score (CRPS), and demonstrates that under certain conditions, the CRPS can be mathematically expressed in terms of the RMSE when these metrics are aggregated. One of the required conditions is the normality of distributions. The other condition is that, while the forecast ensemble need not be calibrated, any bias or over/underdispersion cannot depend on the forecast distribution itself. Under these conditions, the CRPS is a fraction of the RMSE, and this fraction depends only on the heteroscedasticity of the ensemble spread and the measures of calibration. The derived CRPS-RMSE relationship for the case of perfect ensemble reliability is tested on simulations of idealised two-dimensional barotropic turbulence. Results suggest that the relationship holds approximately despite the normality condition not being met. KW - CRPS KW - ensembles KW - idealised turbulence KW - NWP KW - RMSE KW - verification Y1 - 2021 U6 - https://doi.org/10.1002/qj.4120 SN - 0035-9009 SN - 1477-870X VL - 147 IS - 739 SP - 3124 EP - 3134 PB - Wiley CY - Hoboken ER - TY - JOUR A1 - Reich, Sebastian A1 - Weissmann, Simon T1 - Fokker-Planck particle systems for Bayesian inference: computational approaches JF - SIAM ASA journal on uncertainty quantification N2 - Bayesian inference can be embedded into an appropriately defined dynamics in the space of probability measures. In this paper, we take Brownian motion and its associated Fokker-Planck equation as a starting point for such embeddings and explore several interacting particle approximations. More specifically, we consider both deterministic and stochastic interacting particle systems and combine them with the idea of preconditioning by the empirical covariance matrix. In addition to leading to affine invariant formulations which asymptotically speed up convergence, preconditioning allows for gradient-free implementations in the spirit of the ensemble Kalman filter. While such gradient-free implementations have been demonstrated to work well for posterior measures that are nearly Gaussian, we extend their scope of applicability to multimodal measures by introducing localized gradient-free approximations. Numerical results demonstrate the effectiveness of the considered methodologies. KW - Bayesian inverse problems KW - Fokker-Planck equation KW - gradient flow KW - affine KW - invariance KW - gradient-free sampling methods KW - localization Y1 - 2021 U6 - https://doi.org/10.1137/19M1303162 SN - 2166-2525 VL - 9 IS - 2 SP - 446 EP - 482 PB - Society for Industrial and Applied Mathematics CY - Philadelphia ER - TY - JOUR A1 - Pathiraja, Sahani Darschika A1 - Reich, Sebastian A1 - Stannat, Wilhelm T1 - McKean-Vlasov SDEs in nonlinear filtering JF - SIAM journal on control and optimization : a publication of the Society for Industrial and Applied Mathematics N2 - Various particle filters have been proposed over the last couple of decades with the common feature that the update step is governed by a type of control law. This feature makes them an attractive alternative to traditional sequential Monte Carlo which scales poorly with the state dimension due to weight degeneracy. This article proposes a unifying framework that allows us to systematically derive the McKean-Vlasov representations of these filters for the discrete time and continuous time observation case, taking inspiration from the smooth approximation of the data considered in [D. Crisan and J. Xiong, Stochastics, 82 (2010), pp. 53-68; J. M. Clark and D. Crisan, Probab. Theory Related Fields, 133 (2005), pp. 43-56]. We consider three filters that have been proposed in the literature and use this framework to derive Ito representations of their limiting forms as the approximation parameter delta -> 0. All filters require the solution of a Poisson equation defined on R-d, for which existence and uniqueness of solutions can be a nontrivial issue. We additionally establish conditions on the signal-observation system that ensures well-posedness of the weighted Poisson equation arising in one of the filters. KW - data assimilation KW - feedback particle filter KW - Poincare inequality KW - well-posedness KW - nonlinear filtering KW - McKean-Vlasov KW - mean-field equations Y1 - 2022 U6 - https://doi.org/10.1137/20M1355197 SN - 0363-0129 SN - 1095-7138 VL - 59 IS - 6 SP - 4188 EP - 4215 PB - Society for Industrial and Applied Mathematics CY - Philadelphia ER - TY - JOUR A1 - Hastermann, Gottfried A1 - Reinhardt, Maria A1 - Klein, Rupert A1 - Reich, Sebastian T1 - Balanced data assimilation for highly oscillatory mechanical systems JF - Communications in applied mathematics and computational science : CAMCoS N2 - Data assimilation algorithms are used to estimate the states of a dynamical system using partial and noisy observations. The ensemble Kalman filter has become a popular data assimilation scheme due to its simplicity and robustness for a wide range of application areas. Nevertheless, this filter also has limitations due to its inherent assumptions of Gaussianity and linearity, which can manifest themselves in the form of dynamically inconsistent state estimates. This issue is investigated here for balanced, slowly evolving solutions to highly oscillatory Hamiltonian systems which are prototypical for applications in numerical weather prediction. It is demonstrated that the standard ensemble Kalman filter can lead to state estimates that do not satisfy the pertinent balance relations and ultimately lead to filter divergence. Two remedies are proposed, one in terms of blended asymptotically consistent time-stepping schemes, and one in terms of minimization-based postprocessing methods. The effects of these modifications to the standard ensemble Kalman filter are discussed and demonstrated numerically for balanced motions of two prototypical Hamiltonian reference systems. KW - data assimilation KW - ensemble Kalman filter KW - balanced dynamics KW - highly KW - oscillatory systems KW - Hamiltonian dynamics KW - geophysics Y1 - 2021 U6 - https://doi.org/10.2140/camcos.2021.16.119 SN - 1559-3940 SN - 2157-5452 VL - 16 IS - 1 SP - 119 EP - 154 PB - Mathematical Sciences Publishers CY - Berkeley ER - TY - JOUR A1 - Gottwald, Georg A. A1 - Reich, Sebastian T1 - Supervised learning from noisy observations BT - Combining machine-learning techniques with data assimilation JF - Physica : D, Nonlinear phenomena N2 - Data-driven prediction and physics-agnostic machine-learning methods have attracted increased interest in recent years achieving forecast horizons going well beyond those to be expected for chaotic dynamical systems. In a separate strand of research data-assimilation has been successfully used to optimally combine forecast models and their inherent uncertainty with incoming noisy observations. The key idea in our work here is to achieve increased forecast capabilities by judiciously combining machine-learning algorithms and data assimilation. We combine the physics-agnostic data -driven approach of random feature maps as a forecast model within an ensemble Kalman filter data assimilation procedure. The machine-learning model is learned sequentially by incorporating incoming noisy observations. We show that the obtained forecast model has remarkably good forecast skill while being computationally cheap once trained. Going beyond the task of forecasting, we show that our method can be used to generate reliable ensembles for probabilistic forecasting as well as to learn effective model closure in multi-scale systems. (C) 2021 Elsevier B.V. All rights reserved. KW - Data-driven modelling KW - Random feature maps KW - Data assimilation Y1 - 2021 U6 - https://doi.org/10.1016/j.physd.2021.132911 SN - 0167-2789 SN - 1872-8022 VL - 423 PB - Elsevier CY - Amsterdam ER - TY - JOUR A1 - Leung, Tsz Yan A1 - Leutbecher, Martin A1 - Reich, Sebastian A1 - Shepherd, Theodore G. T1 - Impact of the mesoscale range on error growth and the limits to atmospheric predictability JF - Journal of the atmospheric sciences N2 - Global numerical weather prediction (NWP) models have begun to resolve the mesoscale k(-5/3) range of the energy spectrum, which is known to impose an inherently finite range of deterministic predictability per se as errors develop more rapidly on these scales than on the larger scales. However, the dynamics of these errors under the influence of the synoptic-scale k(-3) range is little studied. Within a perfect-model context, the present work examines the error growth behavior under such a hybrid spectrum in Lorenz's original model of 1969, and in a series of identical-twin perturbation experiments using an idealized two-dimensional barotropic turbulence model at a range of resolutions. With the typical resolution of today's global NWP ensembles, error growth remains largely uniform across scales. The theoretically expected fast error growth characteristic of a k(-5/3) spectrum is seen to be largely suppressed in the first decade of the mesoscale range by the synoptic-scale k(-3) range. However, it emerges once models become fully able to resolve features on something like a 20-km scale, which corresponds to a grid resolution on the order of a few kilometers. KW - mesoscale forecasting KW - numerical weather prediction/forecasting KW - short-range prediction KW - numerical analysis/modeling Y1 - 2020 U6 - https://doi.org/10.1175/JAS-D-19-0346.1 SN - 0022-4928 SN - 1520-0469 VL - 77 IS - 11 SP - 3769 EP - 3779 PB - American Meteorological Soc. CY - Boston ER - TY - JOUR A1 - Gottwald, Georg A. A1 - Reich, Sebastian T1 - Combining machine learning and data assimilation to forecast dynamical systems from noisy partial observations JF - Chaos : an interdisciplinary journal of nonlinear science N2 - We present a supervised learning method to learn the propagator map of a dynamical system from partial and noisy observations. In our computationally cheap and easy-to-implement framework, a neural network consisting of random feature maps is trained sequentially by incoming observations within a data assimilation procedure. By employing Takens's embedding theorem, the network is trained on delay coordinates. We show that the combination of random feature maps and data assimilation, called RAFDA, outperforms standard random feature maps for which the dynamics is learned using batch data. Y1 - 2021 U6 - https://doi.org/10.1063/5.0066080 SN - 1054-1500 SN - 1089-7682 VL - 31 IS - 10 PB - AIP CY - Melville ER - TY - JOUR A1 - Maoutsa, Dimitra A1 - Reich, Sebastian A1 - Opper, Manfred T1 - Interacting particle solutions of Fokker–Planck equations through gradient–log–density estimation JF - Entropy N2 - Fokker-Planck equations are extensively employed in various scientific fields as they characterise the behaviour of stochastic systems at the level of probability density functions. Although broadly used, they allow for analytical treatment only in limited settings, and often it is inevitable to resort to numerical solutions. Here, we develop a computational approach for simulating the time evolution of Fokker-Planck solutions in terms of a mean field limit of an interacting particle system. The interactions between particles are determined by the gradient of the logarithm of the particle density, approximated here by a novel statistical estimator. The performance of our method shows promising results, with more accurate and less fluctuating statistics compared to direct stochastic simulations of comparable particle number. Taken together, our framework allows for effortless and reliable particle-based simulations of Fokker-Planck equations in low and moderate dimensions. The proposed gradient-log-density estimator is also of independent interest, for example, in the context of optimal control. KW - stochastic systems KW - Fokker-Planck equation KW - interacting particles KW - multiplicative noise KW - gradient flow KW - stochastic differential equations Y1 - 2020 U6 - https://doi.org/10.3390/e22080802 SN - 1099-4300 VL - 22 IS - 8 PB - MDPI CY - Basel ER - TY - JOUR A1 - de Wiljes, Jana A1 - Pathiraja, Sahani Darschika A1 - Reich, Sebastian T1 - Ensemble transform algorithms for nonlinear smoothing problems JF - SIAM journal on scientific computing N2 - Several numerical tools designed to overcome the challenges of smoothing in a non-linear and non-Gaussian setting are investigated for a class of particle smoothers. The considered family of smoothers is induced by the class of linear ensemble transform filters which contains classical filters such as the stochastic ensemble Kalman filter, the ensemble square root filter, and the recently introduced nonlinear ensemble transform filter. Further the ensemble transform particle smoother is introduced and particularly highlighted as it is consistent in the particle limit and does not require assumptions with respect to the family of the posterior distribution. The linear update pattern of the considered class of linear ensemble transform smoothers allows one to implement important supplementary techniques such as adaptive spread corrections, hybrid formulations, and localization in order to facilitate their application to complex estimation problems. These additional features are derived and numerically investigated for a sequence of increasingly challenging test problems. KW - data assimilation KW - smoother KW - localization KW - optimal transport KW - adaptive KW - spread correction Y1 - 2019 U6 - https://doi.org/10.1137/19M1239544 SN - 1064-8275 SN - 1095-7197 VL - 42 IS - 1 SP - A87 EP - A114 PB - Society for Industrial and Applied Mathematics CY - Philadelphia ER - TY - JOUR A1 - Seelig, Stefan A. A1 - Rabe, Maximilian Michael A1 - Malem-Shinitski, Noa A1 - Risse, Sarah A1 - Reich, Sebastian A1 - Engbert, Ralf T1 - Bayesian parameter estimation for the SWIFT model of eye-movement control during reading JF - Journal of mathematical psychology N2 - Process-oriented theories of cognition must be evaluated against time-ordered observations. Here we present a representative example for data assimilation of the SWIFT model, a dynamical model of the control of fixation positions and fixation durations during natural reading of single sentences. First, we develop and test an approximate likelihood function of the model, which is a combination of a spatial, pseudo-marginal likelihood and a temporal likelihood obtained by probability density approximation Second, we implement a Bayesian approach to parameter inference using an adaptive Markov chain Monte Carlo procedure. Our results indicate that model parameters can be estimated reliably for individual subjects. We conclude that approximative Bayesian inference represents a considerable step forward for computational models of eye-movement control, where modeling of individual data on the basis of process-based dynamic models has not been possible so far. KW - dynamical models KW - reading KW - eye movements KW - saccades KW - likelihood function KW - Bayesian inference KW - MCMC KW - interindividual differences Y1 - 2020 U6 - https://doi.org/10.1016/j.jmp.2019.102313 SN - 0022-2496 SN - 1096-0880 VL - 95 PB - Elsevier CY - San Diego ER -