Learning Partially Known Stochastic Dynamics with Empirical PAC Bayes
arXiv:2006.09914
Abstract
Neural Stochastic Differential Equations model a dynamical environment with neural nets assigned to their drift and diffusion terms. The high expressive power of their nonlinearity comes at the expense of instability in the identification of the large set of free parameters. This paper presents a recipe to improve the prediction accuracy of such models in three steps: i) accounting for epistemic uncertainty by assuming probabilistic weights, ii) incorporation of partial knowledge on the state dynamics, and iii) training the resultant hybrid model by an objective derived from a PAC-Bayesian generalization bound. We observe in our experiments that this recipe effectively translates partial and noisy prior knowledge into an improved model fit.
arXiv commentAccepted at AISTATS 2021
References in corpus (13)
- Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
- Evidential Deep Learning to Quantify Classification Uncertainty
- Variational Dropout and the Local Reparameterization Trick
- Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data
- A Note on the PAC Bayesian Theorem
- Deep Temporal Sigmoid Belief Networks for Sequence Modeling
- Generalized Variational Inference: Three arguments for deriving new Posteriors
- A State Space Approach for Piecewise-Linear Recurrent Neural Networks for Reconstructing Nonlinear Dynamics from Neural Measurements
- Conditional Neural Processes
- Stochastic Runge-Kutta Accelerates Langevin Monte Carlo and Beyond
- Learning Gaussian Processes by Minimizing PAC-Bayesian Generalization Bounds
- Deep learning with differential Gaussian process flows
- Differential Bayesian Neural Nets