Neural SDEs as Infinite-Dimensional GANs
arXiv:2102.03657
Abstract
Stochastic differential equations (SDEs) are a staple of mathematical modelling of temporal dynamics. However, a fundamental limitation has been that such models have typically been relatively inflexible, which recent work introducing Neural SDEs has sought to solve. Here, we show that the current classical approach to fitting SDEs may be approached as a special case of (Wasserstein) GANs, and in doing so the neural and classical regimes may be brought together. The input noise is Brownian motion, the output samples are time-evolving paths produced by a numerical solver, and by parameterising a discriminator as a Neural Controlled Differential Equation (CDE), we obtain Neural SDEs as (in modern machine learning parlance) continuous-time generative time series models. Unlike previous work on this problem, this is a direct extension of the classical approach without reference to either prespecified statistics or density functions. Arbitrary drift and diffusions are admissible, so as the Wasserstein loss has a unique global minima, in the infinite data limit any SDE may be learnt. Example code has been made available as part of the \texttt{torchsde} repository.
arXiv commentPublished at ICML 2021
References in corpus (22)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- ADADELTA: An Adaptive Learning Rate Method
- Spectral Normalization for Generative Adversarial Networks
- Variational Inference with Normalizing Flows
- Generative Modeling by Estimating Gradients of the Data Distribution
- MMD GAN: Towards Deeper Understanding of Moment Matching Network
- Generative Moment Matching Networks
- Real-valued (Medical) Time Series Generation with Recurrent Conditional GANs
- Deep Equilibrium Models
- The Unusual Effectiveness of Averaging in GAN Training
- Dissecting Neural ODEs
- Neural SDE: Stabilizing Neural ODE Networks with Stochastic Noise
- Learning to Detect Sepsis with a Multitask Gaussian Process RNN Classifier
- Learning from the past, predicting the statistics for the future, learning an evolving system
- Signatory: differentiable computations of the signature and logsignature transforms, on both CPU and GPU
- Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit
- A scalable end-to-end Gaussian process adapter for irregularly sampled time series classification
- Stochastic Normalizing Flows
- Statistical Inference for Generative Models with Maximum Mean Discrepancy
- Neural Rough Differential Equations for Long Time Series
- Theoretical guarantees for sampling and inference in generative models with latent diffusions
- Stochasticity in Neural ODEs: An Empirical Study