Evaluating Data Assimilation Algorithms

Abstract

Data assimilation leads naturally to a Bayesian formulation in which the posterior probability distribution of the system state, given the observations, plays a central conceptual role. The aim of this paper is to use this Bayesian posterior probability distribution as a gold standard against which to evaluate various commonly used data assimilation algorithms. A key aspect of geophysical data assimilation is the high dimensionality and low predictability of the computational model. With this in mind, yet with the goal of allowing an explicit and accurate computation of the posterior distribution, we study the 2D Navier-Stokes equations in a periodic geometry. We compute the posterior probability distribution by state-of-the-art statistical sampling techniques. The commonly used algorithms that we evaluate against this accurate gold standard, as quantified by comparing the relative error in reproducing its moments, are 4DVAR and a variety of sequential filtering approximations based on 3DVAR and on extended and ensemble Kalman filters. The primary conclusions are that: (i) with appropriate parameter choices, approximate filters can perform well in reproducing the mean of the desired probability distribution; (ii) however they typically perform poorly when attempting to reproduce the covariance; (iii) this poor performance is compounded by the need to modify the covariance, in order to induce stability. Thus, whilst filters can be a useful tool in predicting mean behavior, they should be viewed with caution as predictors of uncertainty. These conclusions are intrinsic to the algorithms and will not change if the model complexity is increased, for example by employing a smaller viscosity, or by using a detailed NWP model.

References in corpus (3)

• Data assimilation as a nonlinear dynamical systems problem: Stability and convergence of the prediction-assimilation system
• Accuracy and Stability of Filters for Dissipative PDEs
• Stability of Filters for the Navier-Stokes Equation

Cited by in corpus (36)

• Earth System Modeling 2.0: A Blueprint for Models That Learn From Observations and Targeted High-Resolution Simulations
• Quantifying and Reducing Model-Form Uncertainties in Reynolds-Averaged Navier-Stokes Simulations: A Data-Driven, Physics-Based Bayesian Approach
• Can local particle filters beat the curse of dimensionality?
• Data Assimilation: A Mathematical Introduction
• Parameterizations for Ensemble Kalman Inversion
• Digital Twin: Values, Challenges and Enablers
• Hybrid modeling and prediction of dynamical systems
• Deterministic Mean-field Ensemble Kalman Filtering
• Memory embedded non-intrusive reduced order modeling of non-ergodic flows
• Data-Driven CFD Modeling of Turbulent Flows Through Complex Structures
• Data assimilation in the low noise regime with application to the Kuroshio
• Accuracy and Stability of Filters for Dissipative PDEs
• Sampling and resolution characteristics in reduced order models of shallow water equations: intrusive vs non-intrusive
• A comparison of nonlinear extensions to the ensemble Kalman filter: Gaussian Anamorphosis and Two-Step Ensemble Filters
• Quantification of Uncertainties in Turbulence Modeling: A Comparison of Physics-Based and Random Matrix Theoretic Approaches
• Linear theory for filtering nonlinear multiscale systems with model error
• Variational Data Assimilation via Sparse Regularization
• Adaptive Error Covariances Estimation Methods for Ensemble Kalman Filters
• Low-rank statistical finite elements for scalable model-data synthesis
• Information field dynamics for simulation scheme construction
• Regularized Variational Data Assimilation for Bias Treatment using the Wasserstein Metric
• Smoothing and parameter estimation by soft-adherence to governing equations
• Quantum mechanics and data assimilation
• Consistency of maximum likelihood estimation for some dynamical systems
• Uncertainty Quantification of Nonlinear Lagrangian Data Assimilation Using Linear Stochastic Forecast Models
• Randomized maximum likelihood based posterior sampling
• Ensemble Kalman Filters with Resampling
• Stability of Filters for the Navier-Stokes Equation
• On Bayesian data assimilation for PDEs with ill-posed forward problems
• Evaluation of Gaussian approximations for data assimilation in reservoir models
• Generalised Particle Filters with Gaussian Mixtures
• Long-time Asymptotics of the Filtering Distribution for Partially Observed Chaotic Dynamical Systems
• Multi-Index Sequential Monte Carlo Methods for partially observed Stochastic Partial Differential Equations
• Kinetic Methods for Inverse Problems
• 4D Seismic History Matching Incorporating Unsupervised Learning
• Analysis of the 3DVAR Filter for the Partially Observed Lorenz '63 Model