Compositional Modeling of Nonlinear Dynamical Systems with ODE-based Random Features

Abstract

Effectively modeling phenomena present in highly nonlinear dynamical systems whilst also accurately quantifying uncertainty is a challenging task, which often requires problem-specific techniques. We present a novel, domain-agnostic approach to tackling this problem, using compositions of physics-informed random features, derived from ordinary differential equations. The architecture of our model leverages recent advances in approximate inference for deep Gaussian processes, such as layer-wise weight-space approximations which allow us to incorporate random Fourier features, and stochastic variational inference for approximate Bayesian inference. We provide evidence that our model is capable of capturing highly nonlinear behaviour in real-world multivariate time series data. In addition, we find that our approach achieves comparable performance to a number of other probabilistic models on benchmark regression tasks.

arXiv comment19 pages, 6 figures. Accepted to NeurIPS 2021

References in corpus (11)

• Auto-Encoding Variational Bayes
• Decoupled Weight Decay Regularization
• GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration
• Scalable Variational Gaussian Process Classification
• Random Feature Expansions for Deep Gaussian Processes
• Avoiding pathologies in very deep networks
• Inference in Deep Gaussian Processes using Stochastic Gradient Hamiltonian Monte Carlo
• Heterogeneous Multi-output Gaussian Process Prediction
• Deep Gaussian Processes with Importance-Weighted Variational Inference
• Efficient Amortised Bayesian Inference for Hierarchical and Nonlinear Dynamical Systems
• The Gaussian Neural Process