Exploring Machine Speech Chain for Domain Adaptation and Few-Shot Speaker Adaptation
arXiv:2104.03815
Abstract
Machine Speech Chain, which integrates both end-to-end (E2E) automatic speech recognition (ASR) and text-to-speech (TTS) into one circle for joint training, has been proven to be effective in data augmentation by leveraging large amounts of unpaired data. In this paper, we explore the TTS->ASR pipeline in speech chain to do domain adaptation for both neural TTS and E2E ASR models, with only text data from target domain. We conduct experiments by adapting from audiobook domain (LibriSpeech) to presentation domain (TED-LIUM), there is a relative word error rate (WER) reduction of 10% for the E2E ASR model on the TED-LIUM test set, and a relative WER reduction of 51.5% in synthetic speech generated by neural TTS in the presentation domain. Further, we apply few-shot speaker adaptation for the E2E ASR by using a few utterances from target speakers in an unsupervised way, results in additional gains.
References in corpus (12)
- Adam: A Method for Stochastic Optimization
- ADADELTA: An Adaptive Learning Rate Method
- SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
- VoxCeleb2: Deep Speaker Recognition
- VoxCeleb: a large-scale speaker identification dataset
- Attention-Based Models for Speech Recognition
- Sequence Transduction with Recurrent Neural Networks
- FastSpeech: Fast, Robust and Controllable Text to Speech
- Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis
- Listen, Attend and Spell
- Efficiently Trainable Text-to-Speech System Based on Deep Convolutional Networks with Guided Attention
- Machine Speech Chain with One-shot Speaker Adaptation