Transformer with Bidirectional Decoder for Speech Recognition
arXiv:2008.04481
Abstract
Attention-based models have made tremendous progress on end-to-end automatic speech recognition(ASR) recently. However, the conventional transformer-based approaches usually generate the sequence results token by token from left to right, leaving the right-to-left contexts unexploited. In this work, we introduce a bidirectional speech transformer to utilize the different directional contexts simultaneously. Specifically, the outputs of our proposed transformer include a left-to-right target, and a right-to-left target. In inference stage, we use the introduced bidirectional beam search method, which can not only generate left-to-right candidates but also generate right-to-left candidates, and determine the best hypothesis by the score. To demonstrate our proposed speech transformer with a bidirectional decoder(STBD), we conduct extensive experiments on the AISHELL-1 dataset. The results of experiments show that STBD achieves a 3.6\% relative CER reduction(CERR) over the unidirectional speech transformer baseline. Besides, the strongest model in this paper called STBD-Big can achieve 6.64\% CER on the test set, without language model rescoring and any extra data augmentation strategies.
Accepted by InterSpeech 2020
References in corpus (7)
- Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
- Sequence Transduction with Recurrent Neural Networks
- Transformers with convolutional context for ASR
- Improving Transformer-based Speech Recognition Using Unsupervised Pre-training
- Self-Attention Transducers for End-to-End Speech Recognition
- Syllable-Based Sequence-to-Sequence Speech Recognition with the Transformer in Mandarin Chinese
- Independent language modeling architecture for end-to-end ASR