Applying Wav2vec2.0 to Speech Recognition in Various Low-resource Languages

Abstract

There are several domains that own corresponding widely used feature extractors, such as ResNet, BERT, and GPT-x. These models are usually pre-trained on large amounts of unlabeled data by self-supervision and can be effectively applied to downstream tasks. In the speech domain, wav2vec2.0 starts to show its powerful representation ability and feasibility of ultra-low resource speech recognition on the Librispeech corpus, which belongs to the audiobook domain. However, wav2vec2.0 has not been examined on real spoken scenarios and languages other than English. To verify its universality over languages, we apply pre-trained models to solve low-resource speech recognition tasks in various spoken languages. We achieve more than 20% relative improvements in six languages compared with previous work. Among these languages, English achieves a gain of 52.4%. Moreover, using coarse-grained modeling units, such as subword or character, achieves better results than fine-grained modeling units, such as phone or letter.

References in corpus (5)

• Distributed Representations of Words and Phrases and their Compositionality
• Sequence Transduction with Recurrent Neural Networks
• wav2letter++: The Fastest Open-source Speech Recognition System
• Improving Transformer-based Speech Recognition Using Unsupervised Pre-training
• Multilingual End-to-End Speech Recognition with A Single Transformer on Low-Resource Languages

Cited by in corpus (9)

• Towards Better Domain Adaptation for Self-supervised Models: A Case Study of Child ASR
• WESPER: Zero-shot and Realtime Whisper to Normal Voice Conversion for Whisper-based Speech Interactions
• DualVoice: Speech Interaction that Discriminates between Normal and Whispered Voice Input
• End-to-end Speech Translation via Cross-modal Progressive Training
• End-to-end Jordanian dialect speech-to-text self-supervised learning framework
• Automatic recognition of suprasegmentals in speech
• Decoupling recognition and transcription in Mandarin ASR
• Multilingual Zero Resource Speech Recognition Base on Self-Supervise Pre-Trained Acoustic Models
• Label-Synchronous Speech-to-Text Alignment for ASR Using Forward and Backward Transformers