Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data

Abstract

We present a factorized hierarchical variational autoencoder, which learns disentangled and interpretable representations from sequential data without supervision. Specifically, we exploit the multi-scale nature of information in sequential data by formulating it explicitly within a factorized hierarchical graphical model that imposes sequence-dependent priors and sequence-independent priors to different sets of latent variables. The model is evaluated on two speech corpora to demonstrate, qualitatively, its ability to transform speakers or linguistic content by manipulating different sets of latent variables; and quantitatively, its ability to outperform an i-vector baseline for speaker verification and reduce the word error rate by as much as 35% in mismatched train/test scenarios for automatic speech recognition tasks.

Accepted to NIPS 2017

References in corpus (13)

• Adam: A Method for Stochastic Optimization
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• Deep Convolutional Inverse Graphics Network
• Adversarially Learned Inference
• Autoencoding beyond pixels using a learned similarity metric
• Hierarchical Multiscale Recurrent Neural Networks
• Composing graphical models with neural networks for structured representations and fast inference
• Feature Learning in Deep Neural Networks - Studies on Speech Recognition Tasks
• Variational Recurrent Auto-Encoders
• Sequential Neural Models with Stochastic Layers
• Toward Controlled Generation of Text
• Towards a Neural Statistician

Cited by in corpus (31)

• WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
• Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis
• Interpretable and Explainable Machine Learning for Materials Science and Chemistry
• Recent Advances in Autoencoder-Based Representation Learning
• Deep Appearance Models for Face Rendering
• Dynamical Variational Autoencoders: A Comprehensive Review
• Enhancing VAEs for Collaborative Filtering: Flexible Priors & Gating Mechanisms
• Hierarchical Generative Modeling for Controllable Speech Synthesis
• Unsupervised End-to-End Learning of Discrete Linguistic Units for Voice Conversion
• Time-Contrastive Learning Based Deep Bottleneck Features for Text-Dependent Speaker Verification
• Learning Subject-Invariant Representations from Speech-Evoked EEG Using Variational Autoencoders
• TGAVC: Improving Autoencoder Voice Conversion with Text-Guided and Adversarial Training
• Non-Contrastive Self-supervised Learning for Utterance-Level Information Extraction from Speech
• Combining Adversarial Training and Disentangled Speech Representation for Robust Zero-Resource Subword Modeling
• Disentangled Speech Representation Learning Based on Factorized Hierarchical Variational Autoencoder with Self-Supervised Objective
• Sequential Interpretability: Methods, Applications, and Future Direction for Understanding Deep Learning Models in the Context of Sequential Data
• Learning and controlling the source-filter representation of speech with a variational autoencoder
• InfoCatVAE: Representation Learning with Categorical Variational Autoencoders
• Unsupervised TTS Acoustic Modeling for TTS with Conditional Disentangled Sequential VAE
• Improving Unsupervised Subword Modeling via Disentangled Speech Representation Learning and Transformation
• Unsupervised Subword Modeling Using Autoregressive Pretraining and Cross-Lingual Phone-Aware Modeling
• A Brief Overview of Unsupervised Neural Speech Representation Learning
• Hyperprior Induced Unsupervised Disentanglement of Latent Representations
• Disentangling Video with Independent Prediction
• Speech Disorder Classification Using Extended Factorized Hierarchical Variational Auto-encoders
• Discovering Influential Factors in Variational Autoencoders
• Interpreting Models by Allowing to Ask
• A Benchmark of Dynamical Variational Autoencoders applied to Speech Spectrogram Modeling
• Phonetic-and-Semantic Embedding of Spoken Words with Applications in Spoken Content Retrieval
• The pursuit of beauty: Converting image labels to meaningful vectors
• Disentangled Dynamic Representations from Unordered Data