Nonequilibrium Statistical Mechanics and Optimal Prediction of Partially-Observed Complex Systems

Abstract

Only a subset of degrees of freedom are typically accessible or measurable in real-world systems. As a consequence, the proper setting for empirical modeling is that of partially-observed systems. Notably, data-driven models consistently outperform physics-based models for systems with few observable degrees of freedom; e.g., hydrological systems. Here, we provide an operator-theoretic explanation for this empirical success. To predict a partially-observed system's future behavior with physics-based models, the missing degrees of freedom must be explicitly accounted for using data assimilation and model parametrization. Data-driven models, in contrast, employ delay-coordinate embeddings and their evolution under the Koopman operator to implicitly model the effects of the missing degrees of freedom. We describe in detail the statistical physics of partial observations underlying data-driven models using novel Maximum Entropy and Maximum Caliber measures. The resulting nonequilibrium Wiener projections applied to the Mori-Zwanzig formalism reveal how data-driven models may converge to the true dynamics of the observable degrees of freedom. Additionally, this framework shows how data-driven models infer the effects of unobserved degrees of freedom implicitly, in much the same way that physics models infer the effects explicitly. This provides a unified implicit-explicit modeling framework for predicting partially-observed systems, with hybrid physics-informed machine learning methods combining implicit and explicit aspects.

References in corpus (37)

• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• Machine Learning for Fluid Mechanics
• A Data-Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition
• On Dynamic Mode Decomposition: Theory and Applications
• Turbulence Modeling in the Age of Data
• Deep learning for universal linear embeddings of nonlinear dynamics
• Koopman invariant subspaces and finite linear representations of nonlinear dynamical systems for control
• Chaos as an Intermittently Forced Linear System
• Next Generation Reservoir Computing
• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
• Ergodic theory, Dynamic Mode Decomposition and Computation of Spectral Properties of the Koopman operator
• Computational Mechanics: Pattern and Prediction, Structure and Simplicity
• Extended dynamic mode decomposition with dictionary learning: a data-driven adaptive spectral decomposition of the Koopman operator
• The Physics of Climate Variability and Climate Change
• Kernel Mean Embedding of Distributions: A Review and Beyond
• Learning Decentralized Controllers for Robot Swarms with Graph Neural Networks
• On Convergence of Extended Dynamic Mode Decomposition to the Koopman Operator
• Earth System Modeling 2.0: A Blueprint for Models That Learn From Observations and Targeted High-Resolution Simulations
• Data-driven approximation of the Koopman generator: Model reduction, system identification, and control
• On Explaining the Surprising Success of Reservoir Computing Forecaster of Chaos? The Universal Machine Learning Dynamical System with Contrasts to VAR and DMD
• On the numerical approximation of the Perron-Frobenius and Koopman operator
• Data-driven prediction of a multi-scale Lorenz 96 chaotic system using deep learning methods: Reservoir computing, ANN, and RNN-LSTM
• Eigendecompositions of Transfer Operators in Reproducing Kernel Hilbert Spaces
• A discrete approach to stochastic parametrization and dimensional reduction in nonlinear dynamics
• Data-driven model reduction, Wiener projections, and the Koopman-Mori-Zwanzig formalism
• Operator-theoretic framework for forecasting nonlinear time series with kernel analog techniques
• The prediction of future from the past: an old problem from a modern perspective
• Koopman spectra in reproducing kernel Hilbert spaces
• Mori-Zwanzig projection operator formalism for systems with time-dependent Hamiltonians
• Kernel-based Prediction of Non-Markovian Time Series
• Derivation of Transport Equations using the Time-Dependent Projection Operator Method
• Predicting Spatio-Temporal Time Series Using Dimension Reduced Local States
• Two methods to approximate the Koopman operator with a reservoir computer
• Discovering Causal Structure with Reproducing-Kernel Hilbert Space $\epsilon$-Machines
• Inverse Problems and Data Assimilation
• Homeostatic and Adaptive Energetics: Nonequilibrium Fluctuations Beyond Detailed Balance in Voltage-Gated Ion Channels
• Topology, Convergence, and Reconstruction of Predictive States

Cited by in corpus (5)

• A Markovian dynamics for C. elegans behavior across scales
• Maximally predictive states: from partial observations to long timescales
• Fluctuating landscapes and heavy tails in animal behavior
• A reduction scheme for coupled Brownian harmonic oscillators
• Projecting U.S. coastal storm surge risks and impacts with deep learning