The Discovery of Dynamics via Linear Multistep Methods and Deep Learning: Error Estimation

Abstract

Identifying hidden dynamics from observed data is a significant and challenging task in a wide range of applications. Recently, the combination of linear multistep methods (LMMs) and deep learning has been successfully employed to discover dynamics, whereas a complete convergence analysis of this approach is still under development. In this work, we consider the deep network-based LMMs for the discovery of dynamics. We put forward error estimates for these methods using the approximation property of deep networks. It indicates, for certain families of LMMs, that the ${\ell }^2$ grid error is bounded by the sum of $O(h^p)$ and the network approximation error, where $h$ is the time step size and $p$ is the local truncation error order. Numerical results of several physically relevant examples are provided to demonstrate our theory.

References in corpus (28)

• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations
• Neural Tangent Kernel: Convergence and Generalization in Neural Networks
• A Convergence Theory for Deep Learning via Over-Parameterization
• Machine Learning of Linear Differential Equations using Gaussian Processes
• PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network
• A Mean Field View of the Landscape of Two-Layers Neural Networks
• Data Driven Governing Equations Approximation Using Deep Neural Networks
• Deep Network Approximation for Smooth Functions
• Data-Driven Deep Learning of Partial Differential Equations in Modal Space
• On the Spectral Bias of Neural Networks
• Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems
• Neural Network Approximation: Three Hidden Layers Are Enough
• Nonparametric inference of interaction laws in systems of agents from trajectory data
• On the Convergence Rate of Training Recurrent Neural Networks
• Efficient inference of parsimonious phenomenological models of cellular dynamics using S-systems and alternating regression
• Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limit
• Geometry of Optimization and Implicit Regularization in Deep Learning
• Towards Understanding the Spectral Bias of Deep Learning
• Machine Learning for Prediction with Missing Dynamics
• Deep Network with Approximation Error Being Reciprocal of Width to Power of Square Root of Depth
• Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theory
• A comparative study of physics-informed neural network models for learning unknown dynamics and constitutive relations
• How Much Over-parameterization Is Sufficient to Learn Deep ReLU Networks?
• IDENT: Identifying Differential Equations with Numerical Time evolution
• Implicit Regularization of Stochastic Gradient Descent in Natural Language Processing: Observations and Implications
• On the Curse of Memory in Recurrent Neural Networks: Approximation and Optimization Analysis
• FD-Net with Auxiliary Time Steps: Fast Prediction of PDEs using Hessian-Free Trust-Region Methods

Cited by in corpus (3)

• Solving PDEs on Unknown Manifolds with Machine Learning
• Inverse modified differential equations for discovery of dynamics
• Multiscale and Nonlocal Learning for PDEs using Densely Connected RNNs