A Comparison of Discrete Latent Variable Models for Speech Representation Learning

Abstract

Neural latent variable models enable the discovery of interesting structure in speech audio data. This paper presents a comparison of two different approaches which are broadly based on predicting future time-steps or auto-encoding the input signal. Our study compares the representations learned by vq-vae and vq-wav2vec in terms of sub-word unit discovery and phoneme recognition performance. Results show that future time-step prediction with vq-wav2vec achieves better performance. The best system achieves an error rate of 13.22 on the ZeroSpeech 2019 ABX phoneme discrimination challenge

arXiv comment7 pages, 4 figures

References in corpus (9)

• Categorical Reparameterization with Gumbel-Softmax
• Neural Discrete Representation Learning
• SGDR: Stochastic Gradient Descent with Warm Restarts
• vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations
• Learning Discrete Representations via Information Maximizing Self-Augmented Training
• wav2letter++: The Fastest Open-source Speech Recognition System
• Fast Decoding in Sequence Models using Discrete Latent Variables
• The challenge of realistic music generation: modelling raw audio at scale
• Contrastive Predictive Coding Based Feature for Automatic Speaker Verification