Self-Supervised Representations Improve End-to-End Speech Translation
arXiv:2006.12124
Abstract
End-to-end speech-to-text translation can provide a simpler and smaller system but is facing the challenge of data scarcity. Pre-training methods can leverage unlabeled data and have been shown to be effective on data-scarce settings. In this work, we explore whether self-supervised pre-trained speech representations can benefit the speech translation task in both high- and low-resource settings, whether they can transfer well to other languages, and whether they can be effectively combined with other common methods that help improve low-resource end-to-end speech translation such as using a pre-trained high-resource speech recognition system. We demonstrate that self-supervised pre-trained features can consistently improve the translation performance, and cross-lingual transfer allows to extend to a variety of languages without or with little tuning.
arXiv commentAccepted to INTERSPEECH 2020
References in corpus (11)
- Adam: A Method for Stochastic Optimization
- A Simple Framework for Contrastive Learning of Visual Representations
- Representation Learning with Contrastive Predictive Coding
- SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
- vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations
- Common Voice: A Massively-Multilingual Speech Corpus
- fairseq: A Fast, Extensible Toolkit for Sequence Modeling
- Sequence-to-Sequence Models Can Directly Translate Foreign Speech
- Learning Robust and Multilingual Speech Representations
- CoVoST: A Diverse Multilingual Speech-To-Text Translation Corpus
- Multilingual End-to-End Speech Translation