Using Machine Learning to Replicate Chaotic Attractors and Calculate Lyapunov Exponents from Data

Abstract

We use recent advances in the machine learning area known as 'reservoir computing' to formulate a method for model-free estimation from data of the Lyapunov exponents of a chaotic process. The technique uses a limited time series of measurements as input to a high-dimensional dynamical system called a 'reservoir'. After the reservoir's response to the data is recorded, linear regression is used to learn a large set of parameters, called the 'output weights'. The learned output weights are then used to form a modified autonomous reservoir designed to be capable of producing arbitrarily long time series whose ergodic properties approximate those of the input signal. When successful, we say that the autonomous reservoir reproduces the attractor's 'climate'. Since the reservoir equations and output weights are known, we can compute derivatives needed to determine the Lyapunov exponents of the autonomous reservoir, which we then use as estimates of the Lyapunov exponents for the original input generating system. We illustrate the effectiveness of our technique with two examples, the Lorenz system, and the Kuramoto-Sivashinsky (KS) equation. In particular, we use the Lorenz system to show that achieving climate reproduction may require tuning of the reservoir parameters. For the case of the KS equation, we note that as the system's spatial size is increased, the number of Lyapunov exponents increases, thus yielding a challenging test of our method, which we find the method successfully passes.

Cited by in corpus (163)

• Machine learning and the physical sciences
• Next Generation Reservoir Computing
• Attractor Reconstruction by Machine Learning
• Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks
• Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networks
• Hybrid Forecasting of Chaotic Processes: Using Machine Learning in Conjunction with a Knowledge-Based Model
• Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: a case study with the Lorenz 96 model
• Large-Scale Optical Reservoir Computing for Spatiotemporal Chaotic Systems Prediction
• Deep learning of dynamics and signal-noise decomposition with time-stepping constraints
• Data-driven prediction of a multi-scale Lorenz 96 chaotic system using deep learning methods: Reservoir computing, ANN, and RNN-LSTM
• Using machine learning to correct model error in data assimilation and forecast applications
• Forecasting Chaotic Systems with Very Low Connectivity Reservoir Computers
• Embedding and Approximation Theorems for Echo State Networks
• Zermelo's problem: Optimal point-to-point navigation in 2D turbulent flows using Reinforcement Learning
• Using a reservoir computer to learn chaotic attractors, with applications to chaos synchronisation and cryptography
• Data-driven discovery of intrinsic dynamics
• Learning dynamical systems from data: a simple cross-validation perspective
• Deep learning of contagion dynamics on complex networks
• Machine-learning inference of fluid variables from data using reservoir computing
• Adaptive, locally-linear models of complex dynamics
• Good and bad predictions: Assessing and improving the replication of chaotic attractors by means of reservoir computing
• Identifying nonlinear dynamical systems via generative recurrent neural networks with applications to fMRI
• Anticipating synchronization with machine learning
• Classification of chaotic time series with deep learning
• Rapid Time Series Prediction with a Hardware-Based Reservoir Computer
• Selecting embedding delays: An overview of embedding techniques and a new method using persistent homology
• Model-free tracking control of complex dynamical trajectories with machine learning
• Reservoir computing model of two-dimensional turbulent convection
• Long short-term memory embedded nudging schemes for nonlinear data assimilation of geophysical flows
• Integrating Recurrent Neural Networks with Data Assimilation for Scalable Data-Driven State Estimation
• Echo State Networks trained by Tikhonov least squares are L2(μ) approximators of ergodic dynamical systems
• Consistency in Echo-State Networks
• Estimating fractal dimensions: a comparative review and open source implementations
• Chaos on compact manifolds: Differentiable synchronizations beyond the Takens theorem
• Model-free prediction of multistability using echo state network
• Using Machine Learning to Anticipate Tipping Points and Extrapolate to Post-Tipping Dynamics of Non-Stationary Dynamical Systems
• Breaking Symmetries of the Reservoir Equations in Echo State Networks
• Neural Network Forecast of the Sunspot Butterfly Diagram
• Deep learning stochastic processes with QCD phase transition
• Multifunctionality in a Reservoir Computer
• Using reservoir computer to predict and prevent extreme events
• Adaptable Hamiltonian neural networks
• A Review of Mathematical and Computational Methods in Cancer Dynamics
• Dissipation as a resource for Quantum Reservoir Computing
• Learning Hamiltonian dynamics by reservoir computer
• Using Data Assimilation to Train a Hybrid Forecast System that Combines Machine-Learning and Knowledge-Based Components
• Detection of Generalized Synchronization using Echo State Networks
• Accuracy of neural networks for the simulation of chaotic dynamics: precision of training data vs precision of the algorithm
• Learning strange attractors with reservoir systems
• Reservoir Computing Benchmarks: a tutorial review and critique
• One-Shot Learning of Stochastic Differential Equations with Data Adapted Kernels
• Reservoir Computing with Noise
• Finding nonlinear system equations and complex network structures from data: a sparse optimization approach
• Predicting Critical Transitions in Multiscale Dynamical Systems Using Reservoir Computing
• Catch-22s of reservoir computing
• Inferring the dynamics of oscillatory systems using recurrent neural networks
• Stability analysis of chaotic systems from data
• Using Machine Learning to predict extreme events in the Hénon map
• Echo State Networks with Self-Normalizing Activations on the Hyper-Sphere
• Reservoir Computing for Macroeconomic Forecasting with Mixed Frequency Data
• Reducing network size and improving prediction stability of reservoir computing
• Data driven soliton solution of the nonlinear Schrödinger equation with certain $\mathcal{P}\mathcal{T}$-symmetric potentials via deep learning
• Dynamical system analysis of a data-driven model constructed by reservoir computing
• EM-like Learning Chaotic Dynamics from Noisy and Partial Observations
• Model-free prediction of emergence of extreme events in a parametrically driven nonlinear dynamical system by Deep Learning
• Machine Learning Link Inference of Noisy Delay-coupled Networks with Opto-Electronic Experimental Tests
• Dynamical Component Analysis (DyCA): Dimensionality Reduction For High-Dimensional Deterministic Time-Series
• The Role of Data in Model Building and Prediction: A Survey Through Examples
• Risk bounds for reservoir computing
• Chaotic attractor reconstruction using small reservoirs -- the influence of topology
• Criticality in Reservoir Computer of Coupled Phase Oscillators
• SNODE: Spectral Discretization of Neural ODEs for System Identification
• Machine learning with observers predicts complex spatiotemporal behavior
• Echo State Network for two-dimensional turbulent moist Rayleigh-Bénard convection
• Transfer learning of chaotic systems
• Identification of Chimera using Machine Learning
• Approximating the solution to wave propagation using deep neural networks
• Effective models and predictability of chaotic multiscale systems via machine learning
• Reconstruction, forecasting, and stability of chaotic dynamics from partial data
• Learning Continuous Chaotic Attractors with a Reservoir Computer
• Inverse Mechano-Electrical Reconstruction of Cardiac Excitation Wave Patterns from Mechanical Deformation using Deep Learning
• Symmetry-Aware Reservoir Computing
• Reservoir Computing with Generalized Readout based on Generalized Synchronization
• Inferring symbolic dynamics of chaotic flows from persistence
• Supervised machine learning to estimate instabilities in chaotic systems: estimation of local Lyapunov exponents
• Stability Analysis of Reservoir Computers Dynamics via Lyapunov Functions
• Artificial Intelligence, Chaos, Prediction and Understanding in Science
• Curriculum learning for data-driven modeling of dynamical systems
• Surface Similarity Parameter: A New Machine Learning Loss Metric for Oscillatory Spatio-Temporal Data
• Reservoir Computers with Random and Optimized Time-Shifts
• Generalizability of reservoir computing for flux-driven two-dimensional convection
• Learning dynamical systems from data: A simple cross-validation perspective, part III: Irregularly-Sampled Time Series
• Model-assisted deep learning of rare extreme events from partial observations
• Neural Echo State Network using oscillations of gas bubbles in water
• Machine-learning construction of a model for a macroscopic fluid variable using the delay-coordinate of a scalar observable
• Synchronizing Chaos using Reservoir Computing
• Quantum Next Generation Reservoir Computing: An Efficient Quantum Algorithm for Forecasting Quantum Dynamics
• Neural Networks as Geometric Chaotic Maps
• Transition between chaotic and stochastic universality classes of kinetic roughening
• Role of assortativity in predicting burst synchronization using echo state network
• Measuring chaos in the Lorenz and Rössler models: Fidelity tests for reservoir computing
• Predicting positon solutions of a family of Nonlinear Schrödinger equations through Deep Learning algorithm
• Reduced-order adaptive synchronization in a chaotic neural network with parameter mismatch: A dynamical system vs. machine learning approach
• Kernel Methods for the Approximation of the Eigenfunctions of the Koopman Operator
• Assessing observability of chaotic systems using Delay Differential Analysis
• On the Universal Transformation of Data-Driven Models to Control Systems
• Oscillations enhance time-series prediction in reservoir computing with feedback
• Asymmetrically connected reservoir networks learn better
• Chaotic Diffusion of Dissipative Solitons: From Anti-Persistent Random Walk to Hidden Markov Models
• Adaptive Anomaly Detection in Chaotic Time Series with a Spatially Aware Echo State Network
• Real-time forecasting of chaotic dynamics from sparse data and autoencoders
• A Novel Approach for Estimating Largest Lyapunov Exponents in One-Dimensional Chaotic Time Series Using Machine Learning
• Learning the Tangent Space of Dynamical Instabilities from Data
• Physics-Informed Long Short-Term Memory for Forecasting and Reconstruction of Chaos
• Scattering Statistics in Nonlinear Wave Chaotic Systems
• Backpropagation on Dynamical Networks
• Estimating the master stability function from the time series of one oscillator via reservoir computing
• Variational Deep Learning for the Identification and Reconstruction of Chaotic and Stochastic Dynamical Systems from Noisy and Partial Observations
• Learning Dynamics from Noisy Measurements using Deep Learning with a Runge-Kutta Constraint
• How more data can hurt: Instability and regularization in next-generation reservoir computing
• Measuring Lyapunov exponents of large chaotic systems with global coupling by time series analysis
• Hierarchical deep learning-based adaptive time-stepping scheme for multiscale simulations
• Thermodynamic limit in learning period three
• Simple Hamiltonian dynamics is a powerful quantum processing resource
• Universal discrete-time reservoir computers with stochastic inputs and linear readouts using non-homogeneous state-affine systems
• Computing with vortices: Bridging fluid dynamics and its information-processing capability
• Denoising and Reconstruction of Nonlinear Dynamics using Truncated Reservoir Computing
• Differentiable reservoir computing
• Practical Trainable Temporal Postprocessor for Multistate Quantum Measurement
• Parallel BiLSTM-Transformer networks for forecasting chaotic dynamics
• Reservoir Computers Modal Decomposition and Optimization
• STENCIL-NET: Data-driven solution-adaptive discretization of partial differential equations
• Input-dependence in quantum reservoir computing
• Improving the prediction of spatio-temporal chaos by combining parallel reservoir computing with dimensionality reduction
• Tailored minimal reservoir computing: on the bidirectional connection between nonlinearities in the reservoir and in data
• Forecasting the evolution of three-dimensional turbulent recirculating flows from sparse sensor data
• Model-free Forecasting of Rogue Waves using Reservoir Computing
• Predicting three-dimensional chaotic systems with four qubit quantum systems
• Constructing differential equations using only a scalar time-series about continuous time chaotic dynamics
• Neural Lyapunov Model Predictive Control: Learning Safe Global Controllers from Sub-optimal Examples
• Representing ill-known parts of a numerical model using a machine learning approach
• State-space systems as dynamic generative models
• Capturing the Continuous Complexity of Behavior in C. elegans
• Model-free prediction of spatiotemporal dynamical systems with recurrent neural networks: Role of network spectral radius
• Error Bounds of the Invariant Statistics in Machine Learning of Ergodic Itô Diffusions
• Accuracy and Architecture Studies of Residual Neural Network solving Ordinary Differential Equations
• Adapting reservoir computing to solve the Schrödinger equation
• Enhanced Predictive Capability for Chaotic Dynamics by Modified Quantum Reservoir Computing
• Next-Generation Reservoir Computing for Dynamical Inference
• Incorporating Coupling Knowledge into Echo State Networks for Learning Spatiotemporally Chaotic Dynamics
• Long-term prediction of chaotic systems with recurrent neural networks
• Phase transitions from linear to nonlinear information processing in neural networks
• Scope and Sense of Explainability for AI-Systems
• Model-free inference of unseen attractors: Reconstructing phase space features from a single noisy trajectory using reservoir computing
• Invertible generalized synchronization: A putative mechanism for implicit learning in biological and artificial neural systems
• Stability of exoplanetary systems retrieved from scalar time series
• Reservoir observer enhanced with residual calibration and attention mechanism
• Multiclass classification of dephasing channels
• Phase space integrity in neural network models of Hamiltonian dynamics: A Lagrangian descriptor approach
• Finite-time Lyaponov analysis of a trained reservoir computer
• Dynamic Reservoir Computing with Physical Neuromorphic Networks
• 2022 Roadmap on Neuromorphic Computing and Engineering
• Physical reservoir computing using finitely-sampled quantum systems