Towards Online End-to-end Transformer Automatic Speech Recognition

Abstract

The Transformer self-attention network has recently shown promising performance as an alternative to recurrent neural networks in end-to-end (E2E) automatic speech recognition (ASR) systems. However, Transformer has a drawback in that the entire input sequence is required to compute self-attention. We have proposed a block processing method for the Transformer encoder by introducing a context-aware inheritance mechanism. An additional context embedding vector handed over from the previously processed block helps to encode not only local acoustic information but also global linguistic, channel, and speaker attributes. In this paper, we extend it towards an entire online E2E ASR system by introducing an online decoding process inspired by monotonic chunkwise attention (MoChA) into the Transformer decoder. Our novel MoChA training and inference algorithms exploit the unique properties of Transformer, whose attentions are not always monotonic or peaky, and have multiple heads and residual connections of the decoder layers. Evaluations of the Wall Street Journal (WSJ) and AISHELL-1 show that our proposed online Transformer decoder outperforms conventional chunkwise approaches.

arXiv commentarXiv admin note: text overlap with arXiv:1910.07204

References in corpus (9)

• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• Attention-Based Models for Speech Recognition
• Sequence Transduction with Recurrent Neural Networks
• A Comparative Study on Transformer vs RNN in Speech Applications
• Improved training of end-to-end attention models for speech recognition
• Self-Attention Networks for Connectionist Temporal Classification in Speech Recognition
• A Neural Transducer
• Monotonic Chunkwise Attention
• Transformer ASR with Contextual Block Processing

Cited by in corpus (13)

• Online Hybrid CTC/Attention End-to-End Automatic Speech Recognition Architecture
• Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Loss
• Dual-mode ASR: Unify and Improve Streaming ASR with Full-context Modeling
• Low Latency End-to-End Streaming Speech Recognition with a Scout Network
• Streaming Chunk-Aware Multihead Attention for Online End-to-End Speech Recognition
• Gated Recurrent Context: Softmax-free Attention for Online Encoder-Decoder Speech Recognition
• Transformer-based Online Speech Recognition with Decoder-end Adaptive Computation Steps
• Unidirectional Memory-Self-Attention Transducer for Online Speech Recognition
• Head-synchronous Decoding for Transformer-based Streaming ASR
• Improving Streaming Automatic Speech Recognition With Non-Streaming Model Distillation On Unsupervised Data
• Streaming Transformer ASR with Blockwise Synchronous Beam Search
• CarneliNet: Neural Mixture Model for Automatic Speech Recognition
• Streaming Speech-to-Confusion Network Speech Recognition