Statistical Mechanics of Dynamical System Identification

Abstract

Recovering dynamical equations from observed noisy data is the central challenge of system identification. We develop a statistical mechanics approach to analyze sparse equation discovery algorithms, which typically balance data fit and parsimony via hyperparameter tuning. In this framework, statistical mechanics offers tools to analyze the interplay between complexity and fitness similarly to that of entropy and energy in physical systems. To establish this analogy, we define the hyperparameter optimization procedure as a two-level Bayesian inference problem that separates variable selection from coefficient inference and enables the computation of the posterior parameter distribution in closed form. Our approach provides uncertainty quantification, crucial in the low-data limit that is frequently encountered in real-world applications. A key advantage of employing statistical mechanical concepts, such as free energy and the partition function, is to connect the large data limit to thermodynamic limit and characterize the sparsity- and noise-induced phase transitions that delineate correct from incorrect identification. We thus provide a method for closed-loop inference, estimating the noise in a given model and checking if the model is tolerant to that noise amount. This perspective of sparse equation discovery is versatile and can be adapted to various other equation discovery algorithms.

35 RevTeX pages, 17 figures

References in corpus (26)

• Rejoinder to "Least angle regression" by Efron et al
• Least Angle Regression
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discussion of "Least angle regression" by Efron et al
• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• Sparse identification of nonlinear dynamics for model predictive control in the low-data limit
• Statistical physics of inference: Thresholds and algorithms
• Graph spectra and the detectability of community structure in networks
• Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control
• Learning hydrodynamic equations for active matter from particle simulations and experiments
• Numerical Aspects for Approximating Governing Equations Using Data
• On spike-and-slab priors for Bayesian equation discovery of nonlinear dynamical systems via sparse linear regression
• Derivative-based SINDy (DSINDy): Addressing the challenge of discovering governing equations from noisy data
• Direct Estimation of Parameters in ODE Models Using WENDy: Weak-form Estimation of Nonlinear Dynamics
• Data-Driven Discovery of Coarse-Grained Equations
• A Perspective on Regression and Bayesian Approaches for System Identification of Pattern Formation Dynamics
• WeakIdent: Weak formulation for Identifying Differential Equations using Narrow-fit and Trimming
• Data-driven discovery and extrapolation of parameterized pattern-forming dynamics
• Dynamical System Identification, Model Selection and Model Uncertainty Quantification by Bayesian Inference
• Statistical mechanical analysis of sparse linear regression as a variable selection problem