Improving Sequential Latent Variable Models with Autoregressive Flows

Abstract

We propose an approach for improving sequence modeling based on autoregressive normalizing flows. Each autoregressive transform, acting across time, serves as a moving frame of reference, removing temporal correlations, and simplifying the modeling of higher-level dynamics. This technique provides a simple, general-purpose method for improving sequence modeling, with connections to existing and classical techniques. We demonstrate the proposed approach both with standalone flow-based models and as a component within sequential latent variable models. Results are presented on three benchmark video datasets, where autoregressive flow-based dynamics improve log-likelihood performance over baseline models. Finally, we illustrate the decorrelation and improved generalization properties of using flow-based dynamics.

arXiv commentPublished at Machine Learning Journal

References in corpus (38)

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• WaveNet: A Generative Model for Raw Audio
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
• Unsupervised Learning of Video Representations using LSTMs
• NICE: Non-linear Independent Components Estimation
• Training Very Deep Networks
• Variational Inference with Normalizing Flows
• Density estimation using Real NVP
• A Recurrent Latent Variable Model for Sequential Data
• Recurrent World Models Facilitate Policy Evolution
• Parallel WaveNet: Fast High-Fidelity Speech Synthesis
• Learning Latent Dynamics for Planning from Pixels
• Stochastic Video Generation with a Learned Prior
• Masked Autoregressive Flow for Density Estimation
• Learning Stochastic Recurrent Networks
• Covariance Estimation: The GLM and Regularization Perspectives
• Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models
• Glow: Generative Flow with Invertible 1x1 Convolutions
• MoGlow: Probabilistic and controllable motion synthesis using normalising flows
• Sequential Neural Models with Stochastic Layers
• Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks
• Iterative Gaussianization: from ICA to Random Rotations
• Self-Supervised Visual Planning with Temporal Skip Connections
• Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data
• Deep Temporal Sigmoid Belief Networks for Sequence Modeling
• ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech
• High-Dimensional Probability Estimation with Deep Density Models
• Generative Temporal Models with Memory
• Neural Spline Flows
• Deep Generative Video Compression
• Learnable Explicit Density for Continuous Latent Space and Variational Inference
• Transformation Autoregressive Networks
• A General Method for Amortizing Variational Filtering
• Deep Variational Inference Without Pixel-Wise Reconstruction
• Hierarchical Autoregressive Modeling for Neural Video Compression