Conditional Flow Variational Autoencoders for Structured Sequence Prediction

Abstract

Prediction of future states of the environment and interacting agents is a key competence required for autonomous agents to operate successfully in the real world. Prior work for structured sequence prediction based on latent variable models imposes a uni-modal standard Gaussian prior on the latent variables. This induces a strong model bias which makes it challenging to fully capture the multi-modality of the distribution of the future states. In this work, we introduce Conditional Flow Variational Autoencoders (CF-VAE) using our novel conditional normalizing flow based prior to capture complex multi-modal conditional distributions for effective structured sequence prediction. Moreover, we propose two novel regularization schemes which stabilizes training and deals with posterior collapse for stable training and better fit to the target data distribution. Our experiments on three multi-modal structured sequence prediction datasets -- MNIST Sequences, Stanford Drone and HighD -- show that the proposed method obtains state of art results across different evaluation metrics.

arXiv commentTo appear at Bayesian Deep Learning and Machine Learning for Autonomous Driving @NeurIPS 2019

References in corpus (19)

• Auto-Encoding Variational Bayes
• Social Force Model for Pedestrian Dynamics
• NICE: Non-linear Independent Components Estimation
• Variational Inference with Normalizing Flows
• InfoVAE: Information Maximizing Variational Autoencoders
• Variational Lossy Autoencoder
• Variational Approaches for Auto-Encoding Generative Adversarial Networks
• Wasserstein Auto-Encoders
• Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishing
• Analyzing Inverse Problems with Invertible Neural Networks
• DialogWAE: Multimodal Response Generation with Conditional Wasserstein Auto-Encoder
• Improving Variational Auto-Encoders using Householder Flow
• PixelVAE: A Latent Variable Model for Natural Images
• DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents
• Latent Normalizing Flows for Discrete Sequences
• Nonparametric Variational Auto-encoders for Hierarchical Representation Learning
• Accurate and Diverse Sampling of Sequences based on a "Best of Many" Sample Objective
• Scene Induced Multi-Modal Trajectory Forecasting via Planning
• Back to square one: probabilistic trajectory forecasting without bells and whistles

Cited by in corpus (10)

• A Review of Tracking, Prediction and Decision Making Methods for Autonomous Driving
• Trajectory Forecasts in Unknown Environments Conditioned on Grid-Based Plans
• Structured Variational Inference for Simulating Populations of Radio Galaxies
• Clustering multivariate functional data using unsupervised binary trees
• Spectral Temporal Graph Neural Network for Trajectory Prediction
• Imitative Planning using Conditional Normalizing Flow
• Variational Dynamic Mixtures
• Prior Flow Variational Autoencoder: A density estimation model for Non-Intrusive Load Monitoring
• Deep Variational Luenberger-type Observer for Stochastic Video Prediction
• Neural Motion Planning for Autonomous Parking