Learning Robust and Multilingual Speech Representations

Abstract

Unsupervised speech representation learning has shown remarkable success at finding representations that correlate with phonetic structures and improve downstream speech recognition performance. However, most research has been focused on evaluating the representations in terms of their ability to improve the performance of speech recognition systems on read English (e.g. Wall Street Journal and LibriSpeech). This evaluation methodology overlooks two important desiderata that speech representations should have: robustness to domain shifts and transferability to other languages. In this paper we learn representations from up to 8000 hours of diverse and noisy speech data and evaluate the representations by looking at their robustness to domain shifts and their ability to improve recognition performance in many languages. We find that our representations confer significant robustness advantages to the resulting recognition systems: we see significant improvements in out-of-domain transfer relative to baseline feature sets and the features likewise provide improvements in 25 phonetically diverse languages including tonal languages and low-resource languages.

References in corpus (11)

• Distributed Representations of Words and Phrases and their Compositionality
• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• Libri-Light: A Benchmark for ASR with Limited or No Supervision
• MINE: Mutual Information Neural Estimation
• vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations
• Wav2Letter: an End-to-End ConvNet-based Speech Recognition System
• Self-Training for End-to-End Speech Recognition
• End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures
• Mixed-Precision Training for NLP and Speech Recognition with OpenSeq2Seq

Cited by in corpus (7)

• Contrastive Multiview Coding
• BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition
• Multi-Format Contrastive Learning of Audio Representations
• A Further Study of Unsupervised Pre-training for Transformer Based Speech Recognition
• Self-Supervised Representations Improve End-to-End Speech Translation
• Fast Development of ASR in African Languages using Self Supervised Speech Representation Learning
• Representation Learning for Sequence Data with Deep Autoencoding Predictive Components