Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Dataset

Abstract

Recently, Transformer based end-to-end models have achieved great success in many areas including speech recognition. However, compared to LSTM models, the heavy computational cost of the Transformer during inference is a key issue to prevent their applications. In this work, we explored the potential of Transformer Transducer (T-T) models for the fist pass decoding with low latency and fast speed on a large-scale dataset. We combine the idea of Transformer-XL and chunk-wise streaming processing to design a streamable Transformer Transducer model. We demonstrate that T-T outperforms the hybrid model, RNN Transducer (RNN-T), and streamable Transformer attention-based encoder-decoder model in the streaming scenario. Furthermore, the runtime cost and latency can be optimized with a relatively small look-ahead.

arXiv comment5 pages

References in corpus (9)

• Sequence Transduction with Recurrent Neural Networks
• A Comparative Study on Transformer vs RNN in Speech Applications
• Conformer: Convolution-augmented Transformer for Speech Recognition
• Transformer-based Acoustic Modeling for Hybrid Speech Recognition
• Exploring Neural Transducers for End-to-End Speech Recognition
• Transformer-Transducer: End-to-End Speech Recognition with Self-Attention
• RNN-T For Latency Controlled ASR With Improved Beam Search
• Transformer Transducer: One Model Unifying Streaming and Non-streaming Speech Recognition
• Emformer: Efficient Memory Transformer Based Acoustic Model For Low Latency Streaming Speech Recognition

Cited by in corpus (3)

• Streaming end-to-end multi-talker speech recognition
• SimulLR: Simultaneous Lip Reading Transducer with Attention-Guided Adaptive Memory
• Minimum Word Error Rate Training with Language Model Fusion for End-to-End Speech Recognition