The Neural Moving Average Model for Scalable Variational Inference of State Space Models
arXiv:1910.00879
Abstract
Variational inference has had great success in scaling approximate Bayesian inference to big data by exploiting mini-batch training. To date, however, this strategy has been most applicable to models of independent data. We propose an extension to state space models of time series data based on a novel generative model for latent temporal states: the neural moving average model. This permits a subsequence to be sampled without drawing from the entire distribution, enabling training iterations to use mini-batches of the time series at low computational cost. We illustrate our method on autoregressive, Lotka-Volterra, FitzHugh-Nagumo and stochastic volatility models, achieving accurate parameter estimation in a short time.
References in corpus (13)
- Auto-Encoding Variational Bayes
- On the difficulty of training Recurrent Neural Networks
- Conditional Image Generation with PixelCNN Decoders
- Variational Inference with Normalizing Flows
- Fast and scalable Gaussian process modeling with applications to astronomical time series
- Density estimation using Real NVP
- MADE: Masked Autoencoder for Distribution Estimation
- On the efficiency of pseudo-marginal random walk Metropolis algorithms
- Black box variational inference for state space models
- Stochastic Variational Inference for Hidden Markov Models
- Scalable Bayesian Learning for State Space Models using Variational Inference with SMC Samplers
- Online state and parameter estimation in Dynamic Generalised Linear Models
- Variational bridge constructs for approximate Gaussian process regression