Mind the Gap when Conditioning Amortised Inference in Sequential Latent-Variable Models

Abstract

Amortised inference enables scalable learning of sequential latent-variable models (LVMs) with the evidence lower bound (ELBO). In this setting, variational posteriors are often only partially conditioned. While the true posteriors depend, e.g., on the entire sequence of observations, approximate posteriors are only informed by past observations. This mimics the Bayesian filter -- a mixture of smoothing posteriors. Yet, we show that the ELBO objective forces partially-conditioned amortised posteriors to approximate products of smoothing posteriors instead. Consequently, the learned generative model is compromised. We demonstrate these theoretical findings in three scenarios: traffic flow, handwritten digits, and aerial vehicle dynamics. Using fully-conditioned approximate posteriors, performance improves in terms of generative modelling and multi-step prediction.

arXiv commentPublished as a conference paper at ICLR 2021 (Poster)

References in corpus (19)

• Auto-Encoding Variational Bayes
• Rényi Divergence and Kullback-Leibler Divergence
• Variational Inference with Normalizing Flows
• A Recurrent Latent Variable Model for Sequential Data
• Deep Bidirectional and Unidirectional LSTM Recurrent Neural Network for Network-wide Traffic Speed Prediction
• Learning Latent Dynamics for Planning from Pixels
• Stochastic Video Generation with a Learned Prior
• Learning Stochastic Recurrent Networks
• Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks
• Deep Kalman Filters
• Black box variational inference for state space models
• Traffic Graph Convolutional Recurrent Neural Network: A Deep Learning Framework for Network-Scale Traffic Learning and Forecasting
• Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects
• Learning and Querying Fast Generative Models for Reinforcement Learning
• Switching Linear Dynamics for Variational Bayes Filtering
• Learning to Fly via Deep Model-Based Reinforcement Learning
• Approximate Bayesian inference in spatial environments
• Variational Tracking and Prediction with Generative Disentangled State-Space Models
• Re-examination of the Role of Latent Variables in Sequence Modeling