On-the-Fly Aligned Data Augmentation for Sequence-to-Sequence ASR

Abstract

We propose an on-the-fly data augmentation method for automatic speech recognition (ASR) that uses alignment information to generate effective training samples. Our method, called Aligned Data Augmentation (ADA) for ASR, replaces transcribed tokens and the speech representations in an aligned manner to generate previously unseen training pairs. The speech representations are sampled from an audio dictionary that has been extracted from the training corpus and inject speaker variations into the training examples. The transcribed tokens are either predicted by a language model such that the augmented data pairs are semantically close to the original data, or randomly sampled. Both strategies result in training pairs that improve robustness in ASR training. Our experiments on a Seq-to-Seq architecture show that ADA can be applied on top of SpecAugment, and achieves about 9-23% and 4-15% relative improvements in WER over SpecAugment alone on LibriSpeech 100h and LibriSpeech 960h test datasets, respectively.

arXiv commentAccepted at INTERSPEECH 2021

References in corpus (6)

• SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
• A Comparative Study on Transformer vs RNN in Speech Applications
• Self-Training for End-to-End Speech Recognition
• fairseq S2T: Fast Speech-to-Text Modeling with fairseq
• Data Augmentation for End-to-end Code-switching Speech Recognition
• SapAugment: Learning A Sample Adaptive Policy for Data Augmentation

Cited by in corpus (1)

• Make More of Your Data: Minimal Effort Data Augmentation for Automatic Speech Recognition and Translation