Neural Adaptive Sequential Monte Carlo

Abstract

Sequential Monte Carlo (SMC), or particle filtering, is a popular class of methods for sampling from an intractable target distribution using a sequence of simpler intermediate distributions. Like other importance sampling-based methods, performance is critically dependent on the proposal distribution: a bad proposal can lead to arbitrarily inaccurate estimates of the target distribution. This paper presents a new method for automatically adapting the proposal using an approximation of the Kullback-Leibler divergence between the true posterior and the proposal distribution. The method is very flexible, applicable to any parameterized proposal distribution and it supports online and batch variants. We use the new framework to adapt powerful proposal distributions with rich parameterizations based upon neural networks leading to Neural Adaptive Sequential Monte Carlo (NASMC). Experiments indicate that NASMC significantly improves inference in a non-linear state space model outperforming adaptive proposal methods including the Extended Kalman and Unscented Particle Filters. Experiments also indicate that improved inference translates into improved parameter learning when NASMC is used as a subroutine of Particle Marginal Metropolis Hastings. Finally we show that NASMC is able to train a latent variable recurrent neural network (LV-RNN) achieving results that compete with the state-of-the-art for polymorphic music modelling. NASMC can be seen as bridging the gap between adaptive SMC methods and the recent work in scalable, black-box variational inference.

References in corpus (5)

• Sequence to Sequence Learning with Neural Networks
• Expectation Propagation for approximate Bayesian inference
• DRAW: A Recurrent Neural Network For Image Generation
• Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription
• Learning Stochastic Recurrent Networks

Cited by in corpus (18)

• Masked Autoregressive Flow for Density Estimation
• Flexible statistical inference for mechanistic models of neural dynamics
• Learning and Querying Fast Generative Models for Reinforcement Learning
• Inference Networks for Sequential Monte Carlo in Graphical Models
• Variational Sequential Monte Carlo
• ZhuSuan: A Library for Bayesian Deep Learning
• Approximate Inference with Amortised MCMC
• Sequence Tutor: Conservative Fine-Tuning of Sequence Generation Models with KL-control
• Stochastic WaveNet: A Generative Latent Variable Model for Sequential Data
• Bounded Rational Decision-Making with Adaptive Neural Network Priors
• DualSMC: Tunneling Differentiable Filtering and Planning under Continuous POMDPs
• Inference Trees: Adaptive Inference with Exploration
• Data-driven Sequential Monte Carlo in Probabilistic Programming
• Iterative Refinement of the Approximate Posterior for Directed Belief Networks
• Stochastic Sequential Neural Networks with Structured Inference
• Filtering Variational Objectives
• Neural Particle Smoothing for Sampling from Conditional Sequence Models
• Coupled Variational Recurrent Collaborative Filtering