Deep learning for universal linear embeddings of nonlinear dynamics

Abstract

Identifying coordinate transformations that make strongly nonlinear dynamics approximately linear is a central challenge in modern dynamical systems. These transformations have the potential to enable prediction, estimation, and control of nonlinear systems using standard linear theory. The Koopman operator has emerged as a leading data-driven embedding, as eigenfunctions of this operator provide intrinsic coordinates that globally linearize the dynamics. However, identifying and representing these eigenfunctions has proven to be mathematically and computationally challenging. This work leverages the power of deep learning to discover representations of Koopman eigenfunctions from trajectory data of dynamical systems. Our network is parsimonious and interpretable by construction, embedding the dynamics on a low-dimensional manifold that is of the intrinsic rank of the dynamics and parameterized by the Koopman eigenfunctions. In particular, we identify nonlinear coordinates on which the dynamics are globally linear using a modified auto-encoder. We also generalize Koopman representations to include a ubiquitous class of systems that exhibit continuous spectra, ranging from the simple pendulum to nonlinear optics and broadband turbulence. Our framework parametrizes the continuous frequency using an auxiliary network, enabling a compact and efficient embedding at the intrinsic rank, while connecting our models to half a century of asymptotics. In this way, we benefit from the power and generality of deep learning, while retaining the physical interpretability of Koopman embeddings.

arXiv commentv2: added another example and further details (increase from 9 pages to 14 pages and increase from 4 figures to 16 figures)

References in corpus (17)

• Adam: A Method for Stochastic Optimization
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• How transferable are features in deep neural networks?
• A Data-Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition
• On Dynamic Mode Decomposition: Theory and Applications
• Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive control
• Understanding Deep Convolutional Networks
• Applied Koopmanism
• Koopman invariant subspaces and finite linear representations of nonlinear dynamical systems for control
• Chaos as an Intermittently Forced Linear System
• VAMPnets: Deep learning of molecular kinetics
• Ergodic theory, Dynamic Mode Decomposition and Computation of Spectral Properties of the Koopman operator
• Extended dynamic mode decomposition with dictionary learning: a data-driven adaptive spectral decomposition of the Koopman operator
• Time-lagged autoencoders: Deep learning of slow collective variables for molecular kinetics
• Data-driven model reduction and transfer operator approximation
• Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition

Cited by in corpus (272)

• Machine Learning for Fluid Mechanics
• Machine learning accelerated computational fluid dynamics
• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• Explainable Machine Learning for Scientific Insights and Discoveries
• Data-driven discovery of coordinates and governing equations
• Physics-informed learning of governing equations from scarce data
• Enhancing Computational Fluid Dynamics with Machine Learning
• Learning data driven discretizations for partial differential equations
• Discovering physical concepts with neural networks
• Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders
• Data-driven approximation of the Koopman generator: Model reduction, system identification, and control
• Deep Neural Networks with Koopman Operators for Modeling and Control of Autonomous Vehicles
• Deep Neural Networks for Nonlinear Model Order Reduction of Unsteady Flows
• Convolutional neural network based hierarchical autoencoder for nonlinear mode decomposition of fluid field data
• Transformers for Modeling Physical Systems
• Deep learning of dynamics and signal-noise decomposition with time-stepping constraints
• Robust flow field reconstruction from limited measurements via sparse representation
• Discovery of Nonlinear Multiscale Systems: Sampling Strategies and Embeddings
• Data-Driven Modeling and Prediction of Non-Linearizable Dynamics via Spectral Submanifolds
• Enforcing Statistical Constraints in Generative Adversarial Networks for Modeling Chaotic Dynamical Systems
• Deep Learning in Deterministic Computational Mechanics
• Probabilistic neural networks for fluid flow surrogate modeling and data recovery
• Physics-Informed Probabilistic Learning of Linear Embeddings of Non-linear Dynamics With Guaranteed Stability
• Graph Dynamical Networks for Unsupervised Learning of Atomic Scale Dynamics in Materials
• Convolutional neural networks for fluid flow analysis: toward effective metamodeling and low-dimensionalization
• Time-series learning of latent-space dynamics for reduced-order model closure
• Machine Learning on Neutron and X-Ray Scattering
• Hamiltonian neural networks for solving equations of motion
• On Learning Hamiltonian Systems from Data
• Assessment of unsteady flow predictions using hybrid deep learning based reduced order models
• Data-driven discovery of intrinsic dynamics
• Physics-constrained, low-dimensional models for MHD: First-principles and data-driven approaches
• Deep learning to discover and predict dynamics on an inertial manifold
• Uniformly Accurate Machine Learning Based Hydrodynamic Models for Kinetic Equations
• Computer-inspired Quantum Experiments
• Deep learning of contagion dynamics on complex networks
• Physics enhanced neural networks predict order and chaos
• Stabilized Neural Ordinary Differential Equations for Long-Time Forecasting of Dynamical Systems
• Integrated tool-set for Control, Calibration and Characterization of quantum devices applied to superconducting qubits
• Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition
• Reduced order modeling of parametrized systems through autoencoders and SINDy approach: continuation of periodic solutions
• Reconstruction of turbulent data with deep generative models for semantic inpainting from TURB-Rot database
• Learning Parameters and Constitutive Relationships with Physics Informed Deep Neural Networks
• Recovering missing CFD data for high-order discretizations using deep neural networks and dynamics learning
• Active Learning in Robotics: A Review of Control Principles
• Long-term predictions of turbulence by implicit U-Net enhanced Fourier neural operator
• Physics-informed Autoencoders for Lyapunov-stable Fluid Flow Prediction
• Learned Turbulence Modelling with Differentiable Fluid Solvers: Physics-based Loss-functions and Optimisation Horizons
• Attention-Enhanced Neural Network Models for Turbulence Simulation
• A long short-term memory embedding for hybrid uplifted reduced order models
• Feasibility Study of Neural ODE and DAE Modules for Power System Dynamic Component Modeling
• Sparsity-promoting algorithms for the discovery of informative Koopman invariant subspaces
• Single-step deep reinforcement learning for open-loop control of laminar and turbulent flows
• Data-Driven Reduced-Order Modeling of Spatiotemporal Chaos with Neural Ordinary Differential Equations
• Compressed Convolutional LSTM: An Efficient Deep Learning framework to Model High Fidelity 3D Turbulence
• Bluff body uses deep-reinforcement-learning trained active flow control to achieve hydrodynamic stealth
• Koopman mode expansions between simple invariant solutions
• Data-driven Nonlinear Model Reduction to Spectral Submanifolds in Mechanical Systems
• ReduNet: A White-box Deep Network from the Principle of Maximizing Rate Reduction
• Parameterized Neural Ordinary Differential Equations: Applications to Computational Physics Problems
• Learning Compositional Koopman Operators for Model-Based Control
• From Fourier to Koopman: Spectral Methods for Long-term Time Series Prediction
• Embedding Hard Physical Constraints in Neural Network Coarse-Graining of 3D Turbulence
• Deep Learning Enhanced Dynamic Mode Decomposition
• Deep Learning Models for Global Coordinate Transformations that Linearize PDEs
• An advanced hybrid deep adversarial autoencoder for parameterized nonlinear fluid flow modelling
• Riemannian geometry and automatic differentiation for optimization problems of quantum physics and quantum technologies
• Variational approach for learning Markov processes from time series data
• Deep-learning assisted reduced order model for high-dimensional flow prediction from sparse data
• Koopman Operator and its Approximations for Systems with Symmetries
• A Parametric and Feasibility Study for Data Sampling of the Dynamic Mode Decomposition: Spectral Insights and Further Explorations
• Discovering Sparse Interpretable Dynamics from Partial Observations
• Reinforcement-learning-based actuator selection method for active flow control
• Structured Time-Delay Models for Dynamical Systems with Connections to Frenet-Serret Frame
• Deep reconstruction of strange attractors from time series
• Neural modal ordinary differential equations: Integrating physics-based modeling with neural ordinary differential equations for modeling high-dimensional monitored structures
• SE(3) Koopman-MPC: Data-driven Learning and Control of Quadrotor UAVs
• On Some Aspects of the Response to Stochastic and Deterministic Forcings
• Reduced-order Koopman modeling and predictive control of nonlinear processes
• On analytical construction of observable functions in extended dynamic mode decomposition for nonlinear estimation and prediction
• Deep learning probability flows and entropy production rates in active matter
• Koopman based data-driven predictive control
• Manifold embedding data-driven mechanics
• Forecasting Sequential Data using Consistent Koopman Autoencoders
• Scientific intuition inspired by machine learning generated hypotheses
• Predicting Critical Transitions in Multiscale Dynamical Systems Using Reservoir Computing
• Identification of MIMO Wiener-type Koopman Models for Data-Driven Model Reduction using Deep Learning
• Augmenting astrophysical scaling relations with machine learning: application to reducing the Sunyaev-Zeldovich flux-mass scatter
• Neural Canonical Transformation with Symplectic Flows
• Machine learning based non-Newtonian fluid model with molecular fidelity
• Discovering Conservation Laws using Optimal Transport and Manifold Learning
• The Linear-Time-Invariance Notion of the Koopman Analysis-Part 1: The Architecture, Practical Rendering on the Prism Wake, and Fluid-Structure Association
• WeakIdent: Weak formulation for Identifying Differential Equations using Narrow-fit and Trimming
• Probing non-Markovian quantum dynamics with data-driven analysis: Beyond "black-box" machine learning models
• Data-driven Modeling of Rotating Detonation Waves
• Koopman operator learning using invertible neural networks
• Interpretable statistical representations of neural population dynamics and geometry
• Deep Koopman Learning of Nonlinear Time-Varying Systems
• Exploration of Artificial Intelligence-oriented Power System Dynamic Simulators
• Bayesian Inference of Initial Conditions from Non-Linear Cosmic Structures using Field-Level Emulators
• Fast Dynamic 1D Simulation of Divertor Plasmas with Neural PDE Surrogates
• Modal Analysis of Fluid Flows: Applications and Outlook
• Dimensionality reduction to maximize prediction generalization capability
• IDENT: Identifying Differential Equations with Numerical Time evolution
• Compressing fluid flows with nonlinear machine learning: mode decomposition, latent modeling, and flow control
• The Linear-Time-Invariance Notion of the Koopman Analysis-Part 2: Physical Interpretations of Invariant Koopman Modes and Phenomenological Revelations
• Interpretable Conservation Law Estimation by Deriving the Symmetries of Dynamics from Trained Deep Neural Networks
• An overview of Koopman-based control: From error bounds to closed-loop guarantees
• Extracting Forces from Noisy Dynamics in Dusty Plasmas
• Data-driven state-space and Koopman operator models of coherent state dynamics on invariant manifolds
• Dynamics of random pressure fields over bluff bodies: a dynamic mode decomposition perspective
• Symbolic Regression via Neural Networks
• Nonequilibrium Statistical Mechanics and Optimal Prediction of Partially-Observed Complex Systems
• DDK: A Deep Koopman Approach for Dynamics Modeling and Trajectory Tracking of Autonomous Vehicles
• Data-driven Nonlinear Model Reduction using Koopman Theory: Integrated Control Form and NMPC Case Study
• Deeptime: a Python library for machine learning dynamical models from time series data
• Physically-interpretable classification of biological network dynamics for complex collective motions
• Nonlinear Discrete-Time Observers with Physics-Informed Neural Networks
• Estimating asymptotic phase and amplitude functions of limit-cycle oscillators from time series data
• RIBBON: Cost-Effective and QoS-Aware Deep Learning Model Inference using a Diverse Pool of Cloud Computing Instances
• Learning of Causal Observable Functions for Koopman-DFL Lifting Linearization of Nonlinear Controlled Systems and Its Application to Excavation Automation
• Solving PDE-constrained Control Problems Using Operator Learning
• Inexact iterative numerical linear algebra for neural network-based spectral estimation and rare-event prediction
• Two methods to approximate the Koopman operator with a reservoir computer
• Deep Learning of Conjugate Mappings
• Deep Identification of Nonlinear Systems in Koopman Form
• A Physics-Informed Deep Learning Model of the Hot Tail Runaway Electron Seed
• Reconstructing three-dimensional bluff body wake from sectional flow fields with convolutional neural networks
• Combining Dynamic Mode Decomposition with Ensemble Kalman Filtering for Tracking and Forecasting
• Data-driven discovery of multiscale chemical reactions governed by the law of mass action
• Stochastic Deep Koopman Model for Quality Propagation Analysis in Multistage Manufacturing Systems
• System Identification Through Lipschitz Regularized Deep Neural Networks
• Variational Autoencoders for Learning Nonlinear Dynamics of Physical Systems
• Deep learning for nano-photonic materials -- The solution to everything!?
• Slow Invariant Manifolds of Singularly Perturbed Systems via Physics-Informed Machine Learning
• Short note on the behavior of recurrent neural network for noisy dynamical system
• A Novel Modeling Approach for All-Dielectric Metasurfaces Using Deep Neural Networks
• Data-driven feedback stabilization of nonlinear systems: Koopman-based model predictive control
• Bounded nonlinear forecasts of partially observed geophysical systems with physics-constrained deep learning
• Multiresolution Convolutional Autoencoders
• Notes on data-driven output-feedback control of linear MIMO systems
• Density matrix formulation of dynamical systems
• A Tailored Convolutional Neural Network for Nonlinear Manifold Learning of Computational Physics Data using Unstructured Spatial Discretizations
• Self-tuning moving horizon estimation of nonlinear systems via physics-informed machine learning Koopman modeling
• Data-Driven Optimal Control of Tethered Space Robot Deployment with Learning Based Koopman Operator
• Pontryagin Differentiable Programming: An End-to-End Learning and Control Framework
• Fixed-energy inverse scattering with radial basis function neural networks and its application to neutron-alpha interactions
• Memory-Efficient Learning of Stable Linear Dynamical Systems for Prediction and Control
• Adaptive Koopman Embedding for Robust Control of Complex Nonlinear Dynamical Systems
• A Physics-Informed Deep Learning Description of Knudsen Layer Reactivity Reduction
• End-to-End Reinforcement Learning of Koopman Models for Economic Nonlinear Model Predictive Control
• Data-driven reduced-order modeling for nonautonomous dynamical systems in multiscale media
• Statistical properties of large data sets with linear latent features
• tgEDMD: Approximation of the Kolmogorov Operator in Tensor Train Format
• Data-Driven Inference of High-Accuracy Isostable-Based Dynamical Models in Response to External Inputs
• Data-Driven Models for Control Engineering Applications Using the Koopman Operator
• Data-Driven Linear Koopman Embedding for Networked Systems: Model-Predictive Grid Control
• Bridging the Gap: Machine Learning to Resolve Improperly Modeled Dynamics
• Temporally Consistent Koopman Autoencoders for Forecasting Dynamical Systems
• Koopman Operator Dynamical Models: Learning, Analysis and Control
• Combining Federated Learning and Control: A Survey
• Kernel Methods for the Approximation of the Eigenfunctions of the Koopman Operator
• Modeling Partially Observed Nonlinear Dynamical Systems and Efficient Data Assimilation via Discrete-Time Conditional Gaussian Koopman Network
• A physics-informed neural network method for the approximation of slow invariant manifolds for the general class of stiff systems of ODEs
• Scalable Neural Dynamic Equivalence for Power Systems
• Recurrent Neural Networks for Partially Observed Dynamical Systems
• Data-Driven Predictive Control of Nonholonomic Robots Based on a Bilinear Koopman Realization: Data Does Not Replace Geometry
• Data Driven Control with Learned Dynamics: Model-Based versus Model-Free Approach
• Analysis of chaotic dynamical systems with autoencoders
• Promoting global stability in data-driven models of quadratic nonlinear dynamics
• Sampling and Inference of Networked Dynamics using Log-Koopman Nonlinear Graph Fourier Transform
• Extended dynamic mode decomposition with dictionary learning using neural ordinary differential equations
• PCE-PINNs: Physics-Informed Neural Networks for Uncertainty Propagation in Ocean Modeling
• Deep KKL: Data-driven Output Prediction for Non-Linear Systems
• Optimizing Neural Networks via Koopman Operator Theory
• Neural Ordinary Differential Equations for Data-Driven Reduced Order Modeling of Environmental Hydrodynamics
• Stochastic Adversarial Koopman Model for Dynamical Systems
• SPADE4: Sparsity and Delay Embedding based Forecasting of Epidemics
• Deep Learning-based Feature Discovery for Decoding Phenotypic Plasticity in Pediatric High-Grade Gliomas Single-Cell Transcriptomics
• On Koopman Mode Decomposition and Tensor Component Analysis
• Learning Dynamics from Noisy Measurements using Deep Learning with a Runge-Kutta Constraint
• Phase autoencoder for rapid data-driven synchronization of rhythmic spatiotemporal patterns
• Deep Reinforcement Learning in Fluid Mechanics: a promising method for both Active Flow Control and Shape Optimization
• Mitigating Traffic Oscillations in Mixed Traffic Flow with Scalable Deep Koopman Predictive Control
• Hamiltonian Learning using Machine Learning Models Trained with Continuous Measurements
• On learning latent dynamics of the AUG plasma state
• Data-driven control and transfer learning using neural canonical control structures*
• Neural optimal feedback control with local learning rules
• Deep Learning of Koopman Representation for Control
• Discovering time-varying aeroelastic models of a long-span suspension bridge from field measurements by sparse identification of nonlinear dynamical systems
• Deep reinforcement learning for the control of conjugate heat transfer with application to workpiece cooling
• Data-Driven Volumetric Image Generation from Surface Structures using a Patient-Specific Deep Leaning Model
• Towards high-accuracy deep learning inference of compressible turbulent flows over aerofoils
• Multi-Step Deep Koopman Network (MDK-Net) for Vehicle Control in Frenet Frame
• Tensorized Transformer for Dynamical Systems Modeling
• Calibrating multi-dimensional complex ODE from noisy data via deep neural networks
• Pruning deep neural networks generates a sparse, bio-inspired nonlinear controller for insect flight
• Finite Dimensional Koopman Form of Polynomial Nonlinear Systems
• A Convex Optimization Approach to Learning Koopman Operators
• Learning Interpretable Collective Variables for Spreading Processes on Networks
• Koopman Learning with Episodic Memory
• Exploring helical dynamos with machine learning
• Spacetime Autoencoders Using Local Causal States
• Learned Lifted Linearization Applied to Unstable Dynamic Systems Enabled by Koopman Direct Encoding
• Inferring Time-Dependent Distribution Functions from Kinematic Snapshots
• Video Extrapolation with an Invertible Linear Embedding
• Online learning of Koopman operator using streaming data from different dynamical regimes
• Model order reduction with neural networks: Application to laminar and turbulent flows
• A Deep Learning approach for parametrized and time dependent Partial Differential Equations using Dimensionality Reduction and Neural ODEs
• Deep Adversarial Koopman Model for Reaction-Diffusion systems
• Collective variables between large-scale states in turbulent convection
• Painting the Phase Space of Dissipative Systems with Lagrangian Descriptors
• SRoll3: A neural network approach to reduce large-scale systematic effects in the Planck High Frequency Instrument maps
• Physics-enhanced Neural Networks in the Small Data Regime
• Non-intrusive model combination for learning dynamical systems
• Extraction of nonlinearity in neural networks with Koopman operator
• Deep Learning-Aided Model Predictive Control of Wind Farms for AGC Considering the Dynamic Wake Effect
• Suppressing Modulation Instability with Reinforcement Learning
• Learning Dynamics Models with Stable Invariant Sets
• Data-Augmented Predictive Deep Neural Network: Enhancing the extrapolation capabilities of non-intrusive surrogate models
• Transformer-based Koopman Autoencoder for Linearizing Fisher's Equation
• From reductionism to realism: Holistic mathematical modelling for complex biological systems
• Spatiotemporal wall pressure forecast of a rectangular cylinder with physics-aware DeepU-Fourier neural network
• The Effect of Sensor Fusion on Data-Driven Learning of Koopman Operators
• Krylov Subspace Method for Nonlinear Dynamical Systems with Random Noise
• A Neural Network Ensemble Approach to System Identification
• Physics-aware, probabilistic model order reduction with guaranteed stability
• On data-driven stabilization of systems with quadratic nonlinearities
• Kernel Embedding based Variational Approach for Low-dimensional Approximation of Dynamical Systems
• Machine-Learning for Nonintrusive Model Order Reduction of the Parametric Inviscid Transonic Flow past an airfoil
• Assessment of End-to-End and Sequential Data-driven Learning of Fluid Flows
• Surrogate models for Rock-Fluid Interaction: A Grid-Size-Invariant Approach
• Analysis via Orthonormal Systems in Reproducing Kernel Hilbert $C^{\ast }$-Modules and Applications
• Extracting Latent State Representations with Linear Dynamics from Rich Observations
• Simulation of Open Quantum Dynamics with Bootstrap-Based Long Short-Term Memory Recurrent Neural Network
• Density Propagation with Characteristics-based Deep Learning
• DeepGraphDMD: Interpretable Spatio-Temporal Decomposition of Non-linear Functional Brain Network Dynamics
• Optimal Control of Oscillation Timing and Entrainment Using Large Magnitude Inputs: An Adaptive Phase-Amplitude-Coordinate-Based Approach
• Learning Low-Dimensional Quadratic-Embeddings of High-Fidelity Nonlinear Dynamics using Deep Learning
• Generative emulation of chaotic dynamics with coherent prior
• Self-Supervised Decomposition, Disentanglement and Prediction of Video Sequences while Interpreting Dynamics: A Koopman Perspective
• Variational inference formulation for a model-free simulation of a dynamical system with unknown parameters by a recurrent neural network
• Predicting root numbers with neural networks
• Robust Control Design and Analysis Based on Lifting Linearization of Nonlinear Systems Under Uncertain Initial Conditions
• A Survey on Machine Learning Applied to Dynamic Physical Systems
• Learning Stable Koopman Embeddings
• Generative Adversarial Network for Probabilistic Forecast of Random Dynamical System
• Consistent Long-Term Forecasting of Ergodic Dynamical Systems
• Meta-Learning for Koopman Spectral Analysis with Short Time-series
• Generalized Kernel-Based Dynamic Mode Decomposition
• Modeling Electrical Motor Dynamics using Encoder-Decoder with Recurrent Skip Connection
• Data-Driven Continuum Dynamics via Transport-Teleport Duality
• Efficient pseudometrics for data-driven comparisons of nonlinear dynamical systems
• Koopman Operator Based Modeling for Quadrotor Control on $SE(3)$
• Deep Koopman-based Control of Quality Variation in Multistage Manufacturing Systems
• Data-Driven Contact-Aware Control Method for Real-Time Deformable Tool Manipulation: A Case Study in the Environmental Swabbing
• Koopman Eigenfunction-Based Identification and Optimal Nonlinear Control of Turbojet Engine
• Neural network methods for Neumann series problems of Perron-Frobenius operators
• Kalman Filter Aided Federated Koopman Learning
• Improving Robustness to Out-of-Distribution States in Imitation Learning via Deep Koopman-Boosted Diffusion Policy
• Deep Koopman Economic Model Predictive Control of a Pasteurisation Unit
• Higher-Order LaSDI: Reduced Order Modeling with Multiple Time Derivatives
• Reduced-Order Surrogates for Forced Flexible Mesh Coastal-Ocean Models
• Learning Physical Concepts in Cyber-Physical Systems: A Case Study
• Direct data-driven forecast of local turbulent heat flux in Rayleigh-Bénard convection
• Operator Autoencoders: Learning Physical Operations on Encoded Molecular Graphs
• Adaptive Latent Space Tuning for Non-Stationary Distributions
• Extraction of Discrete Spectra Modes from Video Data Using a Deep Convolutional Koopman Network
• Far-Field Minimum-Fuel Spacecraft Rendezvous using Koopman Operator and ${\ell }_2/{\ell }_1$ Optimization
• A purely data-driven framework for prediction, optimization, and control of networked processes: application to networked SIS epidemic model
• Towards Scalable Koopman Operator Learning: Convergence Rates and A Distributed Learning Algorithm
• Supervised DKRC with Images for Offline System Identification