Minimum Bayes Risk Training of RNN-Transducer for End-to-End Speech Recognition

Abstract

In this work, we propose minimum Bayes risk (MBR) training of RNN-Transducer (RNN-T) for end-to-end speech recognition. Specifically, initialized with a RNN-T trained model, MBR training is conducted via minimizing the expected edit distance between the reference label sequence and on-the-fly generated N-best hypothesis. We also introduce a heuristic to incorporate an external neural network language model (NNLM) in RNN-T beam search decoding and explore MBR training with the external NNLM. Experimental results demonstrate an MBR trained model outperforms a RNN-T trained model substantially and further improvements can be achieved if trained with an external NNLM. Our best MBR trained system achieves absolute character error rate (CER) reductions of 1.2% and 0.5% on read and spontaneous Mandarin speech respectively over a strong convolution and transformer based RNN-T baseline trained on ~21,000 hours of speech.

References in corpus (4)

• Sequence Transduction with Recurrent Neural Networks
• Neural Speech Recognizer: Acoustic-to-Word LSTM Model for Large Vocabulary Speech Recognition
• Exploring Architectures, Data and Units For Streaming End-to-End Speech Recognition with RNN-Transducer
• DFSMN-SAN with Persistent Memory Model for Automatic Speech Recognition

Cited by in corpus (7)

• Tied & Reduced RNN-T Decoder
• The 2020 ESPnet update: new features, broadened applications, performance improvements, and future plans
• Research on Modeling Units of Transformer Transducer for Mandarin Speech Recognition
• Minimum Bayes Risk Training for End-to-End Speaker-Attributed ASR
• Improving RNN Transducer Based ASR with Auxiliary Tasks
• Efficient minimum word error rate training of RNN-Transducer for end-to-end speech recognition
• Minimum Word Error Rate Training with Language Model Fusion for End-to-End Speech Recognition