Improving Sequential Latent Variable Models with Autoregressive Flows

Abstract

We propose an approach for improving sequence modeling based on autoregressive normalizing flows. Each autoregressive transform, acting across time, serves as a moving frame of reference, removing temporal correlations, and simplifying the modeling of higher-level dynamics. This technique provides a simple, general-purpose method for improving sequence modeling, with connections to existing and classical techniques. We demonstrate the proposed approach both with standalone flow-based models and as a component within sequential latent variable models. Results are presented on three benchmark video datasets, where autoregressive flow-based dynamics improve log-likelihood performance over baseline models. Finally, we illustrate the decorrelation and improved generalization properties of using flow-based dynamics.

Published at Machine Learning Journal

References in corpus (17)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• WaveNet: A Generative Model for Raw Audio
• NICE: Non-linear Independent Components Estimation
• Parallel WaveNet: Fast High-Fidelity Speech Synthesis
• Learning Latent Dynamics for Planning from Pixels
• Learning Stochastic Recurrent Networks
• Covariance Estimation: The GLM and Regularization Perspectives
• MoGlow: Probabilistic and controllable motion synthesis using normalising flows
• Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks
• Self-Supervised Visual Planning with Temporal Skip Connections
• High-Dimensional Probability Estimation with Deep Density Models
• Generative Temporal Models with Memory
• Neural Spline Flows
• Learnable Explicit Density for Continuous Latent Space and Variational Inference
• A General Method for Amortizing Variational Filtering
• Hierarchical Autoregressive Modeling for Neural Video Compression
• Deep Variational Inference Without Pixel-Wise Reconstruction