TY - JOUR A1 - Reich, Sebastian T1 - A dynamical systems framework for intermittent data assimilation JF - BIT : numerical mathematics ; the leading applied mathematics journal for all computational mathematicians N2 - We consider the problem of discrete time filtering (intermittent data assimilation) for differential equation models and discuss methods for its numerical approximation. The focus is on methods based on ensemble/particle techniques and on the ensemble Kalman filter technique in particular. We summarize as well as extend recent work on continuous ensemble Kalman filter formulations, which provide a concise dynamical systems formulation of the combined dynamics-assimilation problem. Possible extensions to fully nonlinear ensemble/particle based filters are also outlined using the framework of optimal transportation theory. KW - Data assimilation KW - Ensemble Kalman filter KW - Dynamical systems KW - Nonlinear filters KW - Optimal transportation Y1 - 2011 U6 - https://doi.org/10.1007/s10543-010-0302-4 SN - 0006-3835 VL - 51 IS - 1 SP - 235 EP - 249 PB - Springer CY - Dordrecht ER - TY - JOUR A1 - Acevedo, Walter A1 - Reich, Sebastian A1 - Cubasch, Ulrich T1 - Towards the assimilation of tree-ring-width records using ensemble Kalman filtering techniques JF - Climate dynamics : observational, theoretical and computational research on the climate system N2 - This paper investigates the applicability of the Vaganov–Shashkin–Lite (VSL) forward model for tree-ring-width chronologies as observation operator within a proxy data assimilation (DA) setting. Based on the principle of limiting factors, VSL combines temperature and moisture time series in a nonlinear fashion to obtain simulated TRW chronologies. When used as observation operator, this modelling approach implies three compounding, challenging features: (1) time averaging, (2) “switching recording” of 2 variables and (3) bounded response windows leading to “thresholded response”. We generate pseudo-TRW observations from a chaotic 2-scale dynamical system, used as a cartoon of the atmosphere-land system, and attempt to assimilate them via ensemble Kalman filtering techniques. Results within our simplified setting reveal that VSL’s nonlinearities may lead to considerable loss of assimilation skill, as compared to the utilization of a time-averaged (TA) linear observation operator. In order to understand this undesired effect, we embed VSL’s formulation into the framework of fuzzy logic (FL) theory, which thereby exposes multiple representations of the principle of limiting factors. DA experiments employing three alternative growth rate functions disclose a strong link between the lack of smoothness of the growth rate function and the loss of optimality in the estimate of the TA state. Accordingly, VSL’s performance as observation operator can be enhanced by resorting to smoother FL representations of the principle of limiting factors. This finding fosters new interpretations of tree-ring-growth limitation processes. KW - Proxy forward modeling KW - Data assimilation KW - Fuzzy logic KW - Ensemble Kalman filter KW - Paleoclimate reconstruction Y1 - 2016 U6 - https://doi.org/10.1007/s00382-015-2683-1 SN - 0930-7575 SN - 1432-0894 VL - 46 SP - 1909 EP - 1920 PB - Springer CY - New York ER - TY - JOUR A1 - Cucchi, Karma A1 - Hesse, Falk A1 - Kawa, Nura A1 - Wang, Changhong A1 - Rubin, Yoram T1 - Ex-situ priors: A Bayesian hierarchical framework for defining informative prior distributions in hydrogeology JF - Advances in water resources N2 - Stochastic modeling is a common practice for modeling uncertainty in hydrogeology. In stochastic modeling, aquifer properties are characterized by their probability density functions (PDFs). The Bayesian approach for inverse modeling is often used to assimilate information from field measurements collected at a site into properties’ posterior PDFs. This necessitates the definition of a prior PDF, characterizing the knowledge of hydrological properties before undertaking any investigation at the site, and usually coming from previous studies at similar sites. In this paper, we introduce a Bayesian hierarchical algorithm capable of assimilating various information–like point measurements, bounds and moments–into a single, informative PDF that we call ex-situ prior. This informative PDF summarizes the ex-situ information available about a hydrogeological parameter at a site of interest, which can then be used as a prior PDF in future studies at the site. We demonstrate the behavior of the algorithm on several synthetic case studies, compare it to other methods described in the literature, and illustrate the approach by applying it to a public open-access hydrogeological dataset. KW - Data assimilation KW - Data fusion KW - Bayesian hierarchical model KW - Informative prior KW - Databases Y1 - 2019 U6 - https://doi.org/10.1016/j.advwatres.2019.02.003 SN - 0309-1708 SN - 1872-9657 VL - 126 SP - 65 EP - 78 PB - Elsevier CY - Oxford ER - TY - JOUR A1 - Sanchez, Sabrina A1 - Wicht, Johannes A1 - Bärenzung, Julien T1 - Predictions of the geomagnetic secular variation based on the ensemble sequential assimilation of geomagnetic field models by dynamo simulations JF - Earth, planets and space N2 - The IGRF offers an important incentive for testing algorithms predicting the Earth's magnetic field changes, known as secular variation (SV), in a 5-year range. Here, we present a SV candidate model for the 13th IGRF that stems from a sequential ensemble data assimilation approach (EnKF). The ensemble consists of a number of parallel-running 3D-dynamo simulations. The assimilated data are geomagnetic field snapshots covering the years 1840 to 2000 from the COV-OBS.x1 model and for 2001 to 2020 from the Kalmag model. A spectral covariance localization method, considering the couplings between spherical harmonics of the same equatorial symmetry and same azimuthal wave number, allows decreasing the ensemble size to about a 100 while maintaining the stability of the assimilation. The quality of 5-year predictions is tested for the past two decades. These tests show that the assimilation scheme is able to reconstruct the overall SV evolution. They also suggest that a better 5-year forecast is obtained keeping the SV constant compared to the dynamically evolving SV. However, the quality of the dynamical forecast steadily improves over the full assimilation window (180 years). We therefore propose the instantaneous SV estimate for 2020 from our assimilation as a candidate model for the IGRF-13. The ensemble approach provides uncertainty estimates, which closely match the residual differences with respect to the IGRF-13. Longer term predictions for the evolution of the main magnetic field features over a 50-year range are also presented. We observe the further decrease of the axial dipole at a mean rate of 8 nT/year as well as a deepening and broadening of the South Atlantic Anomaly. The magnetic dip poles are seen to approach an eccentric dipole configuration. KW - Earth's magnetic field KW - Geomagnetic secular variation KW - Dynamo KW - simulations KW - Data assimilation Y1 - 2020 U6 - https://doi.org/10.1186/s40623-020-01279-y SN - 1880-5981 VL - 72 IS - 1 PB - Springer CY - New York ER - TY - THES A1 - Zhelavskaya, Irina T1 - Modeling of the Plasmasphere Dynamics T1 - Modellierung der Plasmasphärendynamik N2 - The plasmasphere is a dynamic region of cold, dense plasma surrounding the Earth. Its shape and size are highly susceptible to variations in solar and geomagnetic conditions. Having an accurate model of plasma density in the plasmasphere is important for GNSS navigation and for predicting hazardous effects of radiation in space on spacecraft. The distribution of cold plasma and its dynamic dependence on solar wind and geomagnetic conditions remain, however, poorly quantified. Existing empirical models of plasma density tend to be oversimplified as they are based on statistical averages over static parameters. Understanding the global dynamics of the plasmasphere using observations from space remains a challenge, as existing density measurements are sparse and limited to locations where satellites can provide in-situ observations. In this dissertation, we demonstrate how such sparse electron density measurements can be used to reconstruct the global electron density distribution in the plasmasphere and capture its dynamic dependence on solar wind and geomagnetic conditions. First, we develop an automated algorithm to determine the electron density from in-situ measurements of the electric field on the Van Allen Probes spacecraft. In particular, we design a neural network to infer the upper hybrid resonance frequency from the dynamic spectrograms obtained with the Electric and Magnetic Field Instrument Suite and Integrated Science (EMFISIS) instrumentation suite, which is then used to calculate the electron number density. The developed Neural-network-based Upper hybrid Resonance Determination (NURD) algorithm is applied to more than four years of EMFISIS measurements to produce the publicly available electron density data set. We utilize the obtained electron density data set to develop a new global model of plasma density by employing a neural network-based modeling approach. In addition to the location, the model takes the time history of geomagnetic indices and location as inputs, and produces electron density in the equatorial plane as an output. It is extensively validated using in-situ density measurements from the Van Allen Probes mission, and also by comparing the predicted global evolution of the plasmasphere with the global IMAGE EUV images of He+ distribution. The model successfully reproduces erosion of the plasmasphere on the night side as well as plume formation and evolution, and agrees well with data. The performance of neural networks strongly depends on the availability of training data, which is limited during intervals of high geomagnetic activity. In order to provide reliable density predictions during such intervals, we can employ physics-based modeling. We develop a new approach for optimally combining the neural network- and physics-based models of the plasmasphere by means of data assimilation. The developed approach utilizes advantages of both neural network- and physics-based modeling and produces reliable global plasma density reconstructions for quiet, disturbed, and extreme geomagnetic conditions. Finally, we extend the developed machine learning-based tools and apply them to another important problem in the field of space weather, the prediction of the geomagnetic index Kp. The Kp index is one of the most widely used indicators for space weather alerts and serves as input to various models, such as for the thermosphere, the radiation belts and the plasmasphere. It is therefore crucial to predict the Kp index accurately. Previous work in this area has mostly employed artificial neural networks to nowcast and make short-term predictions of Kp, basing their inferences on the recent history of Kp and solar wind measurements at L1. We analyze how the performance of neural networks compares to other machine learning algorithms for nowcasting and forecasting Kp for up to 12 hours ahead. Additionally, we investigate several machine learning and information theory methods for selecting the optimal inputs to a predictive model of Kp. The developed tools for feature selection can also be applied to other problems in space physics in order to reduce the input dimensionality and identify the most important drivers. Research outlined in this dissertation clearly demonstrates that machine learning tools can be used to develop empirical models from sparse data and also can be used to understand the underlying physical processes. Combining machine learning, physics-based modeling and data assimilation allows us to develop novel methods benefiting from these different approaches. N2 - Die Plasmasphäre ist eine die Erde umgebende dynamische Region aus kaltem, dichtem Plasma. Ihre Form und Größe sind sehr anfällig für Schwankungen der solaren und geomagnetischen Bedingungen. Ein präzises Modell der Plasmadichte in der Plasmasphäre ist wichtig für die GNSS-Navigation und für die Vorhersage gefährlicher Auswirkungen der kosmischen Strahlung auf Raumfahrzeuge. Die Verteilung des kalten Plasmas und seine dynamische Abhängigkeit vom Sonnenwind und den geomagnetischen Bedingungen sind jedoch nach wie vor nur unzureichend quantifiziert. Bestehende empirische Modelle der Plasmadichte sind in der Regel zu stark vereinfacht, da sie auf statistischen Durchschnittswerten statischer Parameter basieren. Das Verständnis der globalen Dynamik der Plasmasphäre anhand von Beobachtungen aus dem Weltraum bleibt eine Herausforderung, da vorhandene Dichtemessungen spärlich sind und sich auf Orte beschränken, an denen Satelliten In-situ-Beobachtungen liefern können. In dieser Dissertation zeigen wir, wie solche spärlichen Elektronendichtemessungen verwendet werden können, um die globale Elektronendichteverteilung in der Plasmasphäre zu rekonstruieren und ihre dynamische Abhängigkeit vom Sonnenwind und den geomagnetischen Bedingungen zu erfassen. Zunächst entwickeln wir einen automatisierten Algorithmus zur Bestimmung der Elektronendichte aus In-situ-Messungen des elektrischen Feldes der Van Allen Probes Raumsonden. Insbesondere entwerfen wir ein neuronales Netzwerk, um die obere Hybridresonanzfrequenz aus den dynamischen Spektrogrammen abzuleiten, die wir durch die Instrumentensuite „Electric and Magnetic Field Instrument Suite“ (EMFISIS) erhielten, welche dann zur Berechnung der Elektronenzahldichte verwendet wird. Der entwickelte „Neural-network-based Upper Hybrid Resonance Determination“ (NURD)-Algorithmus wird auf mehr als vier Jahre der EMFISIS-Messungen angewendet, um den öffentlich verfügbaren Elektronendichte-Datensatz zu erstellen. Wir verwenden den erhaltenen Elektronendichte-Datensatz, um ein neues globales Modell der Plasmadichte zu entwickeln, indem wir einen auf einem neuronalen Netzwerk basierenden Modellierungsansatz verwenden. Zusätzlich zum Ort nimmt das Modell den zeitlichen Verlauf der geomagnetischen Indizes und des Ortes als Eingabe und erzeugt als Ausgabe die Elektronendichte in der äquatorialebene. Dies wird ausführlich anhand von In-situ-Dichtemessungen der Van Allen Probes-Mission und durch den Vergleich der vom Modell vorhergesagten globalen Entwicklung der Plasmasphäre mit den globalen IMAGE EUV-Bildern der He+ -Verteilung validiert. Das Modell reproduziert erfolgreich die Erosion der Plasmasphäre auf der Nachtseite sowie die Bildung und Entwicklung von Fahnen und stimmt gut mit den Daten überein. Die Leistung neuronaler Netze hängt stark von der Verfügbarkeit von Trainingsdaten ab, die für Intervalle hoher geomagnetischer Aktivität nur spärlich vorhanden sind. Um zuverlässige Dichtevorhersagen während solcher Intervalle zu liefern, können wir eine physikalische Modellierung verwenden. Wir entwickeln einen neuen Ansatz zur optimalen Kombination der neuronalen Netzwerk- und physikbasierenden Modelle der Plasmasphäre mittels Datenassimilation. Der entwickelte Ansatz nutzt sowohl die Vorteile neuronaler Netze als auch die physikalischen Modellierung und liefert zuverlässige Rekonstruktionen der globalen Plasmadichte für ruhige, gestörte und extreme geomagnetische Bedingungen. Schließlich erweitern wir die entwickelten auf maschinellem Lernen basierten Werkzeuge und wenden sie auf ein weiteres wichtiges Problem im Bereich des Weltraumwetters an, die Vorhersage des geomagnetischen Index Kp. Der Kp-Index ist einer der am häufigsten verwendeten Indikatoren für Weltraumwetterwarnungen und dient als Eingabe für verschiedene Modelle, z.B. für die Thermosphäre, die Strahlungsgürtel und die Plasmasphäre. Es ist daher wichtig, den Kp-Index genau vorherzusagen. Frühere Arbeiten in diesem Bereich verwendeten hauptsächlich künstliche neuronale Netze, um Kurzzeit-Kp-Vorhersagen zu treffen, wobei deren Schlussfolgerungen auf der jüngsten Vergangenheit von Kp- und Sonnenwindmessungen am L1-Punkt beruhten. Wir analysieren, wie sich die Leistung neuronaler Netze im Vergleich zu anderen Algorithmen für maschinelles Lernen verhält, um kurz- und längerfristige Kp-Voraussagen von bis zu 12 Stunden treffen zu können. Zusätzlich untersuchen wir verschiedene Methoden des maschinellen Lernens und der Informationstheorie zur Auswahl der optimalen Eingaben für ein Vorhersagemodell von Kp. Die entwickelten Werkzeuge zur Merkmalsauswahl können auch auf andere Probleme in der Weltraumphysik angewendet werden, um die Eingabedimensionalität zu reduzieren und die wichtigsten Treiber zu identifizieren. Die in dieser Dissertation skizzierten Untersuchungen zeigen deutlich, dass Werkzeuge für maschinelles Lernen sowohl zur Entwicklung empirischer Modelle aus spärlichen Daten als auch zum Verstehen zugrunde liegender physikalischer Prozesse genutzt werden können. Die Kombination von maschinellem Lernen, physikbasierter Modellierung und Datenassimilation ermöglicht es uns, kombinierte Methoden zu entwickeln, die von unterschiedlichen Ansätzen profitieren. KW - Plasmasphere KW - Inner magnetosphere KW - Neural networks KW - Machine learning KW - Modeling KW - Kp index KW - Geomagnetic activity KW - Data assimilation KW - Validation KW - IMAGE EUV KW - Kalman filter KW - Plasmasphäre KW - Innere Magnetosphäre KW - Neuronale Netze KW - Maschinelles Lernen KW - Modellieren KW - Forecasting KW - Kp-Index KW - Geomagnetische Aktivität KW - Datenassimilation KW - Validierung KW - Kalman Filter KW - Prognose Y1 - 2020 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:517-opus4-482433 ER - TY - JOUR A1 - Gottwald, Georg A. A1 - Reich, Sebastian T1 - Supervised learning from noisy observations BT - Combining machine-learning techniques with data assimilation JF - Physica : D, Nonlinear phenomena N2 - Data-driven prediction and physics-agnostic machine-learning methods have attracted increased interest in recent years achieving forecast horizons going well beyond those to be expected for chaotic dynamical systems. In a separate strand of research data-assimilation has been successfully used to optimally combine forecast models and their inherent uncertainty with incoming noisy observations. The key idea in our work here is to achieve increased forecast capabilities by judiciously combining machine-learning algorithms and data assimilation. We combine the physics-agnostic data -driven approach of random feature maps as a forecast model within an ensemble Kalman filter data assimilation procedure. The machine-learning model is learned sequentially by incorporating incoming noisy observations. We show that the obtained forecast model has remarkably good forecast skill while being computationally cheap once trained. Going beyond the task of forecasting, we show that our method can be used to generate reliable ensembles for probabilistic forecasting as well as to learn effective model closure in multi-scale systems. (C) 2021 Elsevier B.V. All rights reserved. KW - Data-driven modelling KW - Random feature maps KW - Data assimilation Y1 - 2021 U6 - https://doi.org/10.1016/j.physd.2021.132911 SN - 0167-2789 SN - 1872-8022 VL - 423 PB - Elsevier CY - Amsterdam ER -