A Neural Network Ensemble Approach to System Identification

Abstract

We present a new algorithm for learning unknown governing equations from trajectory data, using and ensemble of neural networks. Given samples of solutions $x(t)$ to an unknown dynamical system $\dot{x}(t)=f(t,x(t))$, we approximate the function $f$ using an ensemble of neural networks. We express the equation in integral form and use Euler method to predict the solution at every successive time step using at each iteration a different neural network as a prior for $f$. This procedure yields M-1 time-independent networks, where M is the number of time steps at which $x(t)$ is observed. Finally, we obtain a single function $f(t,x(t))$ by neural network interpolation. Unlike our earlier work, where we numerically computed the derivatives of data, and used them as target in a Lipschitz regularized neural network to approximate $f$, our new method avoids numerical differentiations, which are unstable in presence of noise. We test the new algorithm on multiple examples both with and without noise in the data. We empirically show that generalization and recovery of the governing equation improve by adding a Lipschitz regularization term in our loss function and that this method improves our previous one especially in presence of noise, when numerical differentiation provides low quality target data. Finally, we compare our results with the method proposed by Raissi, et al. arXiv:1801.01236 (2018) and with SINDy.

References in corpus (13)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• Deep learning for universal linear embeddings of nonlinear dynamics
• Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations
• Machine Learning of Linear Differential Equations using Gaussian Processes
• Data Driven Governing Equations Approximation Using Deep Neural Networks
• Sparse Dynamics for Partial Differential Equations
• Data-driven discovery of PDEs in complex datasets
• Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems
• Nonlinear Systems Identification Using Deep Dynamic Neural Networks
• On Robust Computation of Koopman Operator and Prediction in Random Dynamical Systems
• Learning Equations for Extrapolation and Control
• System Identification Through Lipschitz Regularized Deep Neural Networks

Cited by in corpus (1)

• Optimal Transport for Parameter Identification of Chaotic Dynamics via Invariant Measures