Acoustic Feature Learning via Deep Variational Canonical Correlation Analysis

Abstract

We study the problem of acoustic feature learning in the setting where we have access to another (non-acoustic) modality for feature learning but not at test time. We use deep variational canonical correlation analysis (VCCA), a recently proposed deep generative method for multi-view representation learning. We also extend VCCA with improved latent variable priors and with adversarial learning. Compared to other techniques for multi-view feature learning, VCCA's advantages include an intuitive latent variable interpretation and a variational lower bound objective that can be trained end-to-end efficiently. We compare VCCA and its extensions with previous feature learning methods on the University of Wisconsin X-ray Microbeam Database, and show that VCCA-based feature learning improves over previous methods for speaker-independent phonetic recognition.

References in corpus (10)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Energy-based Generative Adversarial Network
• Adversarial Feature Learning
• Adversarially Learned Inference
• Representations of language in a model of visually grounded speech signal
• Deep Variational Canonical Correlation Analysis
• Multilingual Models for Compositional Distributed Semantics
• Multi-view Generative Adversarial Networks
• Stochastic Optimization for Deep CCA via Nonlinear Orthogonal Iterations

Cited by in corpus (2)

• Multi-View Data Generation Without View Supervision
• Variational Inference for Deep Probabilistic Canonical Correlation Analysis