Sequential Neural Models with Stochastic Layers

Abstract

How can we efficiently propagate uncertainty in a latent state representation with recurrent neural networks? This paper introduces stochastic recurrent neural networks which glue a deterministic recurrent neural network and a state space model together to form a stochastic and sequential neural generative model. The clear separation of deterministic and stochastic layers allows a structured variational inference network to track the factorization of the model's posterior distribution. By retaining both the nonlinear recursive structure of a recurrent neural network and averaging over the uncertainty in a latent path, like a state space model, we improve the state of the art results on the Blizzard and TIMIT speech modeling data sets by a large margin, while achieving comparable performances to competing methods on polyphonic music modeling.

NIPS 2016

Cited by in corpus (40)

• An Introduction to Variational Autoencoders
• Human-level performance in first-person multiplayer games with population-based deep reinforcement learning
• Dynamical Variational Autoencoders: A Comprehensive Review
• Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data
• adVAE: A self-adversarial variational autoencoder with Gaussian anomaly prior knowledge for anomaly detection
• A Multi-task Deep Learning Architecture for Maritime Surveillance using AIS Data Streams
• A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning
• Hierarchical Implicit Models and Likelihood-Free Variational Inference
• Improved Variational Autoencoders for Text Modeling using Dilated Convolutions
• The unreasonable effectiveness of the forget gate
• Breaking the Softmax Bottleneck: A High-Rank RNN Language Model
• Learning and Querying Fast Generative Models for Reinforcement Learning
• Generating Multi-Agent Trajectories using Programmatic Weak Supervision
• Deep Encoder-Decoder Models for Unsupervised Learning of Controllable Speech Synthesis
• Generative Temporal Models with Memory
• Z-Forcing: Training Stochastic Recurrent Networks
• A Tutorial on Deep Latent Variable Models of Natural Language
• State Space LSTM Models with Particle MCMC Inference
• Noisin: Unbiased Regularization for Recurrent Neural Networks
• Stochastic WaveNet: A Generative Latent Variable Model for Sequential Data
• Deep Factors with Gaussian Processes for Forecasting
• Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
• Bounded nonlinear forecasts of partially observed geophysical systems with physics-constrained deep learning
• Decentralized policy learning with partial observation and mechanical constraints for multiperson modeling
• Forecasting Individualized Disease Trajectories using Interpretable Deep Learning
• A Classifying Variational Autoencoder with Application to Polyphonic Music Generation
• Learning Awareness Models
• Recurrent Neural Networks with Stochastic Layers for Acoustic Novelty Detection
• Variational Bi-LSTMs
• Improving Sequential Latent Variable Models with Autoregressive Flows
• Variational Inference for Data-Efficient Model Learning in POMDPs
• A Brief Overview of Unsupervised Neural Speech Representation Learning
• The Monte Carlo Transformer: a stochastic self-attention model for sequence prediction
• Filtering Variational Objectives
• Stochastic Sequential Neural Networks with Structured Inference
• Investigating and Improving Latent Density Segmentation Models for Aleatoric Uncertainty Quantification in Medical Imaging
• Bidirectional Inference Networks: A Class of Deep Bayesian Networks for Health Profiling
• Latent Variable Algorithms for Multimodal Learning and Sensor Fusion
• Mixture of Dynamical Variational Autoencoders for Multi-Source Trajectory Modeling and Separation
• A Benchmark of Dynamical Variational Autoencoders applied to Speech Spectrogram Modeling