Dynamical System Identification, Model Selection and Model Uncertainty Quantification by Bayesian Inference

Abstract

This study presents a Bayesian maximum \textit{a~posteriori} (MAP) framework for dynamical system identification from time-series data. This is shown to be equivalent to a generalized Tikhonov regularization, providing a rational justification for the choice of the residual and regularization terms, respectively, from the negative logarithms of the likelihood and prior distributions. In addition to the estimation of model coefficients, the Bayesian interpretation gives access to the full apparatus for Bayesian inference, including the ranking of models, the quantification of model uncertainties and the estimation of unknown (nuisance) hyperparameters. Two Bayesian algorithms, joint maximum \textit{a~posteriori} (JMAP) and variational Bayesian approximation (VBA), are compared to the {LASSO, ridge regression and SINDy algorithms for sparse} regression, by application to several dynamical systems with added {Gaussian or Laplace} noise. For multivariate Gaussian likelihood and prior distributions, the Bayesian formulation gives Gaussian posterior and evidence distributions, in which the numerator terms can be expressed in terms of the Mahalanobis distance or ``Gaussian norm'' $||\vec{y}-\hat{\vec{y}}|{|}_{M^{-1}}^2=(\vec{y}-\hat{\vec{y}}{)}^{\top }{M}^{-1}(\vec{y}-\hat{\vec{y}})$, where $\vec{y}$ is a vector variable, $\hat{\vec{y}}$ is its estimator and $M$ is the covariance matrix. The posterior Gaussian norm is shown to provide a robust metric for quantitative model selection {for the different systems and noise models examined.}

Includes supplementary information

References in corpus (14)

• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations
• Koopman invariant subspaces and finite linear representations of nonlinear dynamical systems for control
• Chaos as an Intermittently Forced Linear System
• Sparse identification of nonlinear dynamics for model predictive control in the low-data limit
• Predicting catastrophes in nonlinear dynamical systems by compressive sensing
• Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control
• SINDy-PI: A Robust Algorithm for Parallel Implicit Sparse Identification of Nonlinear Dynamics
• Model selection for dynamical systems via sparse regression and information criteria
• Sparse Identification of Nonlinear Dynamics for Rapid Model Recovery
• Model selection for hybrid dynamical systems via sparse regression
• Finding nonlinear system equations and complex network structures from data: a sparse optimization approach
• Joint reconstruction and segmentation of noisy velocity images as an inverse Navier-Stokes problem
• Rapid Bayesian identification of sparse nonlinear dynamics from scarce and noisy data

Cited by in corpus (1)

• Statistical Mechanics of Dynamical System Identification