One-Shot Learning of Stochastic Differential Equations with Data Adapted Kernels

Abstract

We consider the problem of learning Stochastic Differential Equations of the form $d{X}_t=f(X_t)dt+\sigma (X_t)d{W}_t$ from one sample trajectory. This problem is more challenging than learning deterministic dynamical systems because one sample trajectory only provides indirect information on the unknown functions $f$, $\sigma$, and stochastic process $d{W}_t$ representing the drift, the diffusion, and the stochastic forcing terms, respectively. We propose a method that combines Computational Graph Completion and data adapted kernels learned via a new variant of cross validation. Our approach can be decomposed as follows: (1) Represent the time-increment map $X_t\to X_{t+dt}$ as a Computational Graph in which $f$, $\sigma$ and $d{W}_t$ appear as unknown functions and random variables. (2) Complete the graph (approximate unknown functions and random variables) via Maximum a Posteriori Estimation (given the data) with Gaussian Process (GP) priors on the unknown functions. (3) Learn the covariance functions (kernels) of the GP priors from data with randomized cross-validation. Numerical experiments illustrate the efficacy, robustness, and scope of our method.

arXiv comment22 pages, 21 figures

References in corpus (14)

• Scikit-learn: Machine Learning in Python
• Practical Bayesian Optimization of Machine Learning Algorithms
• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• Using Machine Learning to Replicate Chaotic Attractors and Calculate Lyapunov Exponents from Data
• Gradient-based Hyperparameter Optimization through Reversible Learning
• Analysis of data sets of stochastic systems
• Data-driven approximation of the Koopman generator: Model reduction, system identification, and control
• Kernel Flows: from learning kernels from data into the abyss
• Learning dynamical systems from data: a simple cross-validation perspective
• Operator-theoretic framework for forecasting nonlinear time series with kernel analog techniques
• Kernel-based approximation of the Koopman generator and Schrödinger operator
• Kernel methods for center manifold approximation and a data-based version of the Center Manifold Theorem
• A Note on Kernel Methods for Multiscale Systems with Critical Transitions
• Learning stochastic dynamical systems with neural networks mimicking the Euler-Maruyama scheme

Cited by in corpus (4)

• Learning dynamical systems from data: A simple cross-validation perspective, part III: Irregularly-Sampled Time Series
• Modeling Unknown Stochastic Dynamical System via Autoencoder
• Codiscovering graphical structure and functional relationships within data: A Gaussian Process framework for connecting the dots
• A probabilistic approach to drift estimation from stochastic data