Deep learning for universal linear embeddings of nonlinear dynamics

Abstract

Identifying coordinate transformations that make strongly nonlinear dynamics approximately linear is a central challenge in modern dynamical systems. These transformations have the potential to enable prediction, estimation, and control of nonlinear systems using standard linear theory. The Koopman operator has emerged as a leading data-driven embedding, as eigenfunctions of this operator provide intrinsic coordinates that globally linearize the dynamics. However, identifying and representing these eigenfunctions has proven to be mathematically and computationally challenging. This work leverages the power of deep learning to discover representations of Koopman eigenfunctions from trajectory data of dynamical systems. Our network is parsimonious and interpretable by construction, embedding the dynamics on a low-dimensional manifold that is of the intrinsic rank of the dynamics and parameterized by the Koopman eigenfunctions. In particular, we identify nonlinear coordinates on which the dynamics are globally linear using a modified auto-encoder. We also generalize Koopman representations to include a ubiquitous class of systems that exhibit continuous spectra, ranging from the simple pendulum to nonlinear optics and broadband turbulence. Our framework parametrizes the continuous frequency using an auxiliary network, enabling a compact and efficient embedding at the intrinsic rank, while connecting our models to half a century of asymptotics. In this way, we benefit from the power and generality of deep learning, while retaining the physical interpretability of Koopman embeddings.

v2: added another example and further details (increase from 9 pages to 14 pages and increase from 4 figures to 16 figures)

References in corpus (6)

• How transferable are features in deep neural networks?
• Applied Koopmanism
• VAMPnets: Deep learning of molecular kinetics
• Extended dynamic mode decomposition with dictionary learning: a data-driven adaptive spectral decomposition of the Koopman operator
• Time-lagged autoencoders: Deep learning of slow collective variables for molecular kinetics
• Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition

Cited by in corpus (271)

• Machine Learning for Fluid Mechanics
• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• Machine learning accelerated computational fluid dynamics
• Explainable Machine Learning for Scientific Insights and Discoveries
• Data-driven discovery of coordinates and governing equations
• Physics-informed learning of governing equations from scarce data
• Enhancing Computational Fluid Dynamics with Machine Learning
• Learning data driven discretizations for partial differential equations
• Discovering physical concepts with neural networks
• Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders
• Data-driven approximation of the Koopman generator: Model reduction, system identification, and control
• Deep Neural Networks for Nonlinear Model Order Reduction of Unsteady Flows
• Deep Neural Networks with Koopman Operators for Modeling and Control of Autonomous Vehicles
• Convolutional neural network based hierarchical autoencoder for nonlinear mode decomposition of fluid field data
• Transformers for Modeling Physical Systems
• Deep learning of dynamics and signal-noise decomposition with time-stepping constraints
• Robust flow field reconstruction from limited measurements via sparse representation
• Discovery of Nonlinear Multiscale Systems: Sampling Strategies and Embeddings
• Data-Driven Modeling and Prediction of Non-Linearizable Dynamics via Spectral Submanifolds
• Enforcing Statistical Constraints in Generative Adversarial Networks for Modeling Chaotic Dynamical Systems
• Deep Learning in Deterministic Computational Mechanics
• Probabilistic neural networks for fluid flow surrogate modeling and data recovery
• Physics-Informed Probabilistic Learning of Linear Embeddings of Non-linear Dynamics With Guaranteed Stability
• Graph Dynamical Networks for Unsupervised Learning of Atomic Scale Dynamics in Materials
• Convolutional neural networks for fluid flow analysis: toward effective metamodeling and low-dimensionalization
• Time-series learning of latent-space dynamics for reduced-order model closure
• Machine Learning on Neutron and X-Ray Scattering
• Hamiltonian neural networks for solving equations of motion
• On Learning Hamiltonian Systems from Data
• Assessment of unsteady flow predictions using hybrid deep learning based reduced order models
• Data-driven discovery of intrinsic dynamics
• Physics-constrained, low-dimensional models for MHD: First-principles and data-driven approaches
• Deep learning to discover and predict dynamics on an inertial manifold
• Uniformly Accurate Machine Learning Based Hydrodynamic Models for Kinetic Equations
• Computer-inspired Quantum Experiments
• Deep learning of contagion dynamics on complex networks
• Stabilized Neural Ordinary Differential Equations for Long-Time Forecasting of Dynamical Systems
• Physics enhanced neural networks predict order and chaos
• Integrated tool-set for Control, Calibration and Characterization of quantum devices applied to superconducting qubits
• Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition
• Reconstruction of turbulent data with deep generative models for semantic inpainting from TURB-Rot database
• Reduced order modeling of parametrized systems through autoencoders and SINDy approach: continuation of periodic solutions
• Learning Parameters and Constitutive Relationships with Physics Informed Deep Neural Networks
• Active Learning in Robotics: A Review of Control Principles
• Recovering missing CFD data for high-order discretizations using deep neural networks and dynamics learning
• Long-term predictions of turbulence by implicit U-Net enhanced Fourier neural operator
• Physics-informed Autoencoders for Lyapunov-stable Fluid Flow Prediction
• Learned Turbulence Modelling with Differentiable Fluid Solvers: Physics-based Loss-functions and Optimisation Horizons
• A long short-term memory embedding for hybrid uplifted reduced order models
• Attention-Enhanced Neural Network Models for Turbulence Simulation
• Sparsity-promoting algorithms for the discovery of informative Koopman invariant subspaces
• Single-step deep reinforcement learning for open-loop control of laminar and turbulent flows
• Feasibility Study of Neural ODE and DAE Modules for Power System Dynamic Component Modeling
• Data-Driven Reduced-Order Modeling of Spatiotemporal Chaos with Neural Ordinary Differential Equations
• Compressed Convolutional LSTM: An Efficient Deep Learning framework to Model High Fidelity 3D Turbulence
• Bluff body uses deep-reinforcement-learning trained active flow control to achieve hydrodynamic stealth
• Data-driven Nonlinear Model Reduction to Spectral Submanifolds in Mechanical Systems
• Koopman mode expansions between simple invariant solutions
• ReduNet: A White-box Deep Network from the Principle of Maximizing Rate Reduction
• Parameterized Neural Ordinary Differential Equations: Applications to Computational Physics Problems
• Learning Compositional Koopman Operators for Model-Based Control
• From Fourier to Koopman: Spectral Methods for Long-term Time Series Prediction
• Embedding Hard Physical Constraints in Neural Network Coarse-Graining of 3D Turbulence
• Deep Learning Enhanced Dynamic Mode Decomposition
• An advanced hybrid deep adversarial autoencoder for parameterized nonlinear fluid flow modelling
• Riemannian geometry and automatic differentiation for optimization problems of quantum physics and quantum technologies
• Deep Learning Models for Global Coordinate Transformations that Linearize PDEs
• Deep-learning assisted reduced order model for high-dimensional flow prediction from sparse data
• Koopman Operator and its Approximations for Systems with Symmetries
• A Parametric and Feasibility Study for Data Sampling of the Dynamic Mode Decomposition: Spectral Insights and Further Explorations
• Discovering Sparse Interpretable Dynamics from Partial Observations
• Reinforcement-learning-based actuator selection method for active flow control
• Structured Time-Delay Models for Dynamical Systems with Connections to Frenet-Serret Frame
• Deep reconstruction of strange attractors from time series
• Neural modal ordinary differential equations: Integrating physics-based modeling with neural ordinary differential equations for modeling high-dimensional monitored structures
• SE(3) Koopman-MPC: Data-driven Learning and Control of Quadrotor UAVs
• On analytical construction of observable functions in extended dynamic mode decomposition for nonlinear estimation and prediction
• Reduced-order Koopman modeling and predictive control of nonlinear processes
• On Some Aspects of the Response to Stochastic and Deterministic Forcings
• Forecasting Sequential Data using Consistent Koopman Autoencoders
• Manifold embedding data-driven mechanics
• Scientific intuition inspired by machine learning generated hypotheses
• Koopman based data-driven predictive control
• Deep learning probability flows and entropy production rates in active matter
• Identification of MIMO Wiener-type Koopman Models for Data-Driven Model Reduction using Deep Learning
• Predicting Critical Transitions in Multiscale Dynamical Systems Using Reservoir Computing
• Augmenting astrophysical scaling relations with machine learning: application to reducing the Sunyaev-Zeldovich flux-mass scatter
• Machine learning based non-Newtonian fluid model with molecular fidelity
• Neural Canonical Transformation with Symplectic Flows
• Discovering Conservation Laws using Optimal Transport and Manifold Learning
• The Linear-Time-Invariance Notion of the Koopman Analysis-Part 1: The Architecture, Practical Rendering on the Prism Wake, and Fluid-Structure Association
• WeakIdent: Weak formulation for Identifying Differential Equations using Narrow-fit and Trimming
• Koopman operator learning using invertible neural networks
• Deep Koopman Learning of Nonlinear Time-Varying Systems
• Data-driven Modeling of Rotating Detonation Waves
• Probing non-Markovian quantum dynamics with data-driven analysis: Beyond "black-box" machine learning models
• Interpretable statistical representations of neural population dynamics and geometry
• Exploration of Artificial Intelligence-oriented Power System Dynamic Simulators
• Bayesian Inference of Initial Conditions from Non-Linear Cosmic Structures using Field-Level Emulators
• Modal Analysis of Fluid Flows: Applications and Outlook
• Fast Dynamic 1D Simulation of Divertor Plasmas with Neural PDE Surrogates
• Dimensionality reduction to maximize prediction generalization capability
• IDENT: Identifying Differential Equations with Numerical Time evolution
• The Linear-Time-Invariance Notion of the Koopman Analysis-Part 2: Physical Interpretations of Invariant Koopman Modes and Phenomenological Revelations
• Compressing fluid flows with nonlinear machine learning: mode decomposition, latent modeling, and flow control
• Interpretable Conservation Law Estimation by Deriving the Symmetries of Dynamics from Trained Deep Neural Networks
• An overview of Koopman-based control: From error bounds to closed-loop guarantees
• Data-driven state-space and Koopman operator models of coherent state dynamics on invariant manifolds
• Symbolic Regression via Neural Networks
• Extracting Forces from Noisy Dynamics in Dusty Plasmas
• Dynamics of random pressure fields over bluff bodies: a dynamic mode decomposition perspective
• Nonequilibrium Statistical Mechanics and Optimal Prediction of Partially-Observed Complex Systems
• Nonlinear Discrete-Time Observers with Physics-Informed Neural Networks
• Deeptime: a Python library for machine learning dynamical models from time series data
• Data-driven Nonlinear Model Reduction using Koopman Theory: Integrated Control Form and NMPC Case Study
• Physically-interpretable classification of biological network dynamics for complex collective motions
• DDK: A Deep Koopman Approach for Dynamics Modeling and Trajectory Tracking of Autonomous Vehicles
• Solving PDE-constrained Control Problems Using Operator Learning
• Estimating asymptotic phase and amplitude functions of limit-cycle oscillators from time series data
• Learning of Causal Observable Functions for Koopman-DFL Lifting Linearization of Nonlinear Controlled Systems and Its Application to Excavation Automation
• RIBBON: Cost-Effective and QoS-Aware Deep Learning Model Inference using a Diverse Pool of Cloud Computing Instances
• Two methods to approximate the Koopman operator with a reservoir computer
• Inexact iterative numerical linear algebra for neural network-based spectral estimation and rare-event prediction
• Deep Learning of Conjugate Mappings
• Variational Autoencoders for Learning Nonlinear Dynamics of Physical Systems
• Deep Identification of Nonlinear Systems in Koopman Form
• Reconstructing three-dimensional bluff body wake from sectional flow fields with convolutional neural networks
• System Identification Through Lipschitz Regularized Deep Neural Networks
• Stochastic Deep Koopman Model for Quality Propagation Analysis in Multistage Manufacturing Systems
• Data-driven discovery of multiscale chemical reactions governed by the law of mass action
• A Physics-Informed Deep Learning Model of the Hot Tail Runaway Electron Seed
• Combining Dynamic Mode Decomposition with Ensemble Kalman Filtering for Tracking and Forecasting
• Short note on the behavior of recurrent neural network for noisy dynamical system
• Slow Invariant Manifolds of Singularly Perturbed Systems via Physics-Informed Machine Learning
• A Novel Modeling Approach for All-Dielectric Metasurfaces Using Deep Neural Networks
• Deep learning for nano-photonic materials -- The solution to everything!?
• Notes on data-driven output-feedback control of linear MIMO systems
• Multiresolution Convolutional Autoencoders
• Data-driven feedback stabilization of nonlinear systems: Koopman-based model predictive control
• Density matrix formulation of dynamical systems
• Bounded nonlinear forecasts of partially observed geophysical systems with physics-constrained deep learning
• Data-Driven Optimal Control of Tethered Space Robot Deployment with Learning Based Koopman Operator
• Self-tuning moving horizon estimation of nonlinear systems via physics-informed machine learning Koopman modeling
• A Tailored Convolutional Neural Network for Nonlinear Manifold Learning of Computational Physics Data using Unstructured Spatial Discretizations
• Pontryagin Differentiable Programming: An End-to-End Learning and Control Framework
• Fixed-energy inverse scattering with radial basis function neural networks and its application to neutron-alpha interactions
• End-to-End Reinforcement Learning of Koopman Models for Economic Nonlinear Model Predictive Control
• Statistical properties of large data sets with linear latent features
• Adaptive Koopman Embedding for Robust Control of Complex Nonlinear Dynamical Systems
• Memory-Efficient Learning of Stable Linear Dynamical Systems for Prediction and Control
• Data-driven reduced-order modeling for nonautonomous dynamical systems in multiscale media
• Data-Driven Inference of High-Accuracy Isostable-Based Dynamical Models in Response to External Inputs
• A Physics-Informed Deep Learning Description of Knudsen Layer Reactivity Reduction
• tgEDMD: Approximation of the Kolmogorov Operator in Tensor Train Format
• Combining Federated Learning and Control: A Survey
• Temporally Consistent Koopman Autoencoders for Forecasting Dynamical Systems
• Data-Driven Models for Control Engineering Applications Using the Koopman Operator
• Data-Driven Linear Koopman Embedding for Networked Systems: Model-Predictive Grid Control
• Kernel Methods for the Approximation of the Eigenfunctions of the Koopman Operator
• Koopman Operator Dynamical Models: Learning, Analysis and Control
• Bridging the Gap: Machine Learning to Resolve Improperly Modeled Dynamics
• Scalable Neural Dynamic Equivalence for Power Systems
• Recurrent Neural Networks for Partially Observed Dynamical Systems
• A physics-informed neural network method for the approximation of slow invariant manifolds for the general class of stiff systems of ODEs
• Data-Driven Predictive Control of Nonholonomic Robots Based on a Bilinear Koopman Realization: Data Does Not Replace Geometry
• Modeling Partially Observed Nonlinear Dynamical Systems and Efficient Data Assimilation via Discrete-Time Conditional Gaussian Koopman Network
• Data Driven Control with Learned Dynamics: Model-Based versus Model-Free Approach
• Analysis of chaotic dynamical systems with autoencoders
• Promoting global stability in data-driven models of quadratic nonlinear dynamics
• Sampling and Inference of Networked Dynamics using Log-Koopman Nonlinear Graph Fourier Transform
• Extended dynamic mode decomposition with dictionary learning using neural ordinary differential equations
• SPADE4: Sparsity and Delay Embedding based Forecasting of Epidemics
• Stochastic Adversarial Koopman Model for Dynamical Systems
• PCE-PINNs: Physics-Informed Neural Networks for Uncertainty Propagation in Ocean Modeling
• Deep Learning-based Feature Discovery for Decoding Phenotypic Plasticity in Pediatric High-Grade Gliomas Single-Cell Transcriptomics
• Neural Ordinary Differential Equations for Data-Driven Reduced Order Modeling of Environmental Hydrodynamics
• Learning Dynamics from Noisy Measurements using Deep Learning with a Runge-Kutta Constraint
• Deep KKL: Data-driven Output Prediction for Non-Linear Systems
• Optimizing Neural Networks via Koopman Operator Theory
• On Koopman Mode Decomposition and Tensor Component Analysis
• Mitigating Traffic Oscillations in Mixed Traffic Flow with Scalable Deep Koopman Predictive Control
• On learning latent dynamics of the AUG plasma state
• Hamiltonian Learning using Machine Learning Models Trained with Continuous Measurements
• Data-driven control and transfer learning using neural canonical control structures*
• Discovering time-varying aeroelastic models of a long-span suspension bridge from field measurements by sparse identification of nonlinear dynamical systems
• Neural optimal feedback control with local learning rules
• Deep Learning of Koopman Representation for Control
• Deep Reinforcement Learning in Fluid Mechanics: a promising method for both Active Flow Control and Shape Optimization
• Phase autoencoder for rapid data-driven synchronization of rhythmic spatiotemporal patterns
• Tensorized Transformer for Dynamical Systems Modeling
• Finite Dimensional Koopman Form of Polynomial Nonlinear Systems
• Pruning deep neural networks generates a sparse, bio-inspired nonlinear controller for insect flight
• Towards high-accuracy deep learning inference of compressible turbulent flows over aerofoils
• Calibrating multi-dimensional complex ODE from noisy data via deep neural networks
• Multi-Step Deep Koopman Network (MDK-Net) for Vehicle Control in Frenet Frame
• Data-Driven Volumetric Image Generation from Surface Structures using a Patient-Specific Deep Leaning Model
• Deep reinforcement learning for the control of conjugate heat transfer with application to workpiece cooling
• A Convex Optimization Approach to Learning Koopman Operators
• Learning Interpretable Collective Variables for Spreading Processes on Networks
• Collective variables between large-scale states in turbulent convection
• A Deep Learning approach for parametrized and time dependent Partial Differential Equations using Dimensionality Reduction and Neural ODEs
• Online learning of Koopman operator using streaming data from different dynamical regimes
• Spacetime Autoencoders Using Local Causal States
• Video Extrapolation with an Invertible Linear Embedding
• Exploring helical dynamos with machine learning
• Learned Lifted Linearization Applied to Unstable Dynamic Systems Enabled by Koopman Direct Encoding
• Model order reduction with neural networks: Application to laminar and turbulent flows
• Inferring Time-Dependent Distribution Functions from Kinematic Snapshots
• Koopman Learning with Episodic Memory
• Deep Adversarial Koopman Model for Reaction-Diffusion systems
• Machine-Learning for Nonintrusive Model Order Reduction of the Parametric Inviscid Transonic Flow past an airfoil
• SRoll3: A neural network approach to reduce large-scale systematic effects in the Planck High Frequency Instrument maps
• From reductionism to realism: Holistic mathematical modelling for complex biological systems
• Spatiotemporal wall pressure forecast of a rectangular cylinder with physics-aware DeepU-Fourier neural network
• Deep Learning-Aided Model Predictive Control of Wind Farms for AGC Considering the Dynamic Wake Effect
• Learning Dynamics Models with Stable Invariant Sets
• Painting the Phase Space of Dissipative Systems with Lagrangian Descriptors
• Transformer-based Koopman Autoencoder for Linearizing Fisher's Equation
• Data-Augmented Predictive Deep Neural Network: Enhancing the extrapolation capabilities of non-intrusive surrogate models
• The Effect of Sensor Fusion on Data-Driven Learning of Koopman Operators
• Physics-enhanced Neural Networks in the Small Data Regime
• A Neural Network Ensemble Approach to System Identification
• Suppressing Modulation Instability with Reinforcement Learning
• Physics-aware, probabilistic model order reduction with guaranteed stability
• Krylov Subspace Method for Nonlinear Dynamical Systems with Random Noise
• Assessment of End-to-End and Sequential Data-driven Learning of Fluid Flows
• On data-driven stabilization of systems with quadratic nonlinearities
• Extraction of nonlinearity in neural networks with Koopman operator
• Kernel Embedding based Variational Approach for Low-dimensional Approximation of Dynamical Systems
• Non-intrusive model combination for learning dynamical systems
• Optimal Control of Oscillation Timing and Entrainment Using Large Magnitude Inputs: An Adaptive Phase-Amplitude-Coordinate-Based Approach
• Surrogate models for Rock-Fluid Interaction: A Grid-Size-Invariant Approach
• Self-Supervised Decomposition, Disentanglement and Prediction of Video Sequences while Interpreting Dynamics: A Koopman Perspective
• Generative emulation of chaotic dynamics with coherent prior
• A Survey on Machine Learning Applied to Dynamic Physical Systems
• Predicting root numbers with neural networks
• Robust Control Design and Analysis Based on Lifting Linearization of Nonlinear Systems Under Uncertain Initial Conditions
• DeepGraphDMD: Interpretable Spatio-Temporal Decomposition of Non-linear Functional Brain Network Dynamics
• Simulation of Open Quantum Dynamics with Bootstrap-Based Long Short-Term Memory Recurrent Neural Network
• Learning Low-Dimensional Quadratic-Embeddings of High-Fidelity Nonlinear Dynamics using Deep Learning
• Analysis via Orthonormal Systems in Reproducing Kernel Hilbert $C^{\ast }$-Modules and Applications
• Variational inference formulation for a model-free simulation of a dynamical system with unknown parameters by a recurrent neural network
• Density Propagation with Characteristics-based Deep Learning
• Extracting Latent State Representations with Linear Dynamics from Rich Observations
• Data-Driven Contact-Aware Control Method for Real-Time Deformable Tool Manipulation: A Case Study in the Environmental Swabbing
• Kalman Filter Aided Federated Koopman Learning
• Reduced-Order Surrogates for Forced Flexible Mesh Coastal-Ocean Models
• Data-Driven Continuum Dynamics via Transport-Teleport Duality
• Extraction of Discrete Spectra Modes from Video Data Using a Deep Convolutional Koopman Network
• Efficient pseudometrics for data-driven comparisons of nonlinear dynamical systems
• Koopman Operator Based Modeling for Quadrotor Control on $SE(3)$
• Deep Koopman-based Control of Quality Variation in Multistage Manufacturing Systems
• Higher-Order LaSDI: Reduced Order Modeling with Multiple Time Derivatives
• Learning Physical Concepts in Cyber-Physical Systems: A Case Study
• Direct data-driven forecast of local turbulent heat flux in Rayleigh-Bénard convection
• Operator Autoencoders: Learning Physical Operations on Encoded Molecular Graphs
• Far-Field Minimum-Fuel Spacecraft Rendezvous using Koopman Operator and ${\ell }_2/{\ell }_1$ Optimization
• Adaptive Latent Space Tuning for Non-Stationary Distributions
• A purely data-driven framework for prediction, optimization, and control of networked processes: application to networked SIS epidemic model
• Supervised DKRC with Images for Offline System Identification
• Generative Adversarial Network for Probabilistic Forecast of Random Dynamical System
• Modeling Electrical Motor Dynamics using Encoder-Decoder with Recurrent Skip Connection
• Learning Stable Koopman Embeddings
• Deep Koopman Economic Model Predictive Control of a Pasteurisation Unit
• Neural network methods for Neumann series problems of Perron-Frobenius operators
• Generalized Kernel-Based Dynamic Mode Decomposition
• Towards Scalable Koopman Operator Learning: Convergence Rates and A Distributed Learning Algorithm
• Meta-Learning for Koopman Spectral Analysis with Short Time-series
• Koopman Eigenfunction-Based Identification and Optimal Nonlinear Control of Turbojet Engine
• Consistent Long-Term Forecasting of Ergodic Dynamical Systems
• Improving Robustness to Out-of-Distribution States in Imitation Learning via Deep Koopman-Boosted Diffusion Policy