Variational Recurrent Auto-Encoders

Abstract

In this paper we propose a model that combines the strengths of RNNs and SGVB: the Variational Recurrent Auto-Encoder (VRAE). Such a model can be used for efficient, large scale unsupervised learning on time series data, mapping the time series data to a latent vector representation. The model is generative, such that data can be generated from samples of the latent space. An important contribution of this work is that the model can make use of unlabeled data in order to facilitate supervised training of RNNs by initialising the weights and network state.

Accepted at ICLR workshop track

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Sequence to Sequence Learning with Neural Networks
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription
• Learning Stochastic Recurrent Networks

Cited by in corpus (41)

• A Recurrent Latent Variable Model for Sequential Data
• A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music
• Real-valued (Medical) Time Series Generation with Recurrent Conditional GANs
• Dynamical Variational Autoencoders: A Comprehensive Review
• A Review on Neural Network Models of Schizophrenia and Autism Spectrum Disorder
• Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data
• Bayesian Recurrent Neural Networks
• Statistical Speech Enhancement Based on Probabilistic Integration of Variational Autoencoder and Non-Negative Matrix Factorization
• Deep Temporal Sigmoid Belief Networks for Sequence Modeling
• A Multi-Modal Chinese Poetry Generation Model
• Stochastic WaveNet: A Generative Latent Variable Model for Sequential Data
• A New Multi-vehicle Trajectory Generator to Simulate Vehicle-to-Vehicle Encounters
• PixelVAE++: Improved PixelVAE with Discrete Prior
• Drop, Swap, and Generate: A Self-Supervised Approach for Generating Neural Activity
• A Classifying Variational Autoencoder with Application to Polyphonic Music Generation
• TimeVAE: A Variational Auto-Encoder for Multivariate Time Series Generation
• MIDI-Sandwich2: RNN-based Hierarchical Multi-modal Fusion Generation VAE networks for multi-track symbolic music generation
• Multi-space Variational Encoder-Decoders for Semi-supervised Labeled Sequence Transduction
• Using Large Ensembles of Control Variates for Variational Inference
• S3VAE: Self-Supervised Sequential VAE for Representation Disentanglement and Data Generation
• The NES Music Database: A multi-instrumental dataset with expressive performance attributes
• Semi-Recurrent CNN-based VAE-GAN for Sequential Data Generation
• mu-Forcing: Training Variational Recurrent Autoencoders for Text Generation
• Max-Affine Spline Insights into Deep Generative Networks
• Factored Temporal Sigmoid Belief Networks for Sequence Learning
• Snore-GANs: Improving Automatic Snore Sound Classification with Synthesized Data
• mmFall: Fall Detection using 4D MmWave Radar and a Hybrid Variational RNN AutoEncoder
• The Monte Carlo Transformer: a stochastic self-attention model for sequence prediction
• Stochastic Sequential Neural Networks with Structured Inference
• Likelihood Contribution based Multi-scale Architecture for Generative Flows
• MaGNET: Uniform Sampling from Deep Generative Network Manifolds Without Retraining
• Organizational Artifacts of Code Development
• Driving Style Representation in Convolutional Recurrent Neural Network Model of Driver Identification
• Investigation of Sentiment Controllable Chatbot
• Detecting Patterns of Physiological Response to Hemodynamic Stress via Unsupervised Deep Learning
• The Hierarchical Adaptive Forgetting Variational Filter
• Towards Generating Real-World Time Series Data
• A Benchmark of Dynamical Variational Autoencoders applied to Speech Spectrogram Modeling
• Anomaly Detection of Time Series with Smoothness-Inducing Sequential Variational Auto-Encoder
• Uncertainty-Gated Stochastic Sequential Model for EHR Mortality Prediction
• Modeling Melodic Feature Dependency with Modularized Variational Auto-Encoder