Self-Attention Transducers for End-to-End Speech Recognition

Abstract

Recurrent neural network transducers (RNN-T) have been successfully applied in end-to-end speech recognition. However, the recurrent structure makes it difficult for parallelization . In this paper, we propose a self-attention transducer (SA-T) for speech recognition. RNNs are replaced with self-attention blocks, which are powerful to model long-term dependencies inside sequences and able to be efficiently parallelized. Furthermore, a path-aware regularization is proposed to assist SA-T to learn alignments and improve the performance. Additionally, a chunk-flow mechanism is utilized to achieve online decoding. All experiments are conducted on a Mandarin Chinese dataset AISHELL-1. The results demonstrate that our proposed approach achieves a 21.3% relative reduction in character error rate compared with the baseline RNN-T. In addition, the SA-T with chunk-flow mechanism can perform online decoding with only a little degradation of the performance.

References in corpus (2)

• Sequence Transduction with Recurrent Neural Networks
• Self-Attention Networks for Connectionist Temporal Classification in Speech Recognition

Cited by in corpus (24)

• SpeechFormer++: A Hierarchical Efficient Framework for Paralinguistic Speech Processing
• Exploring Transformers for Large-Scale Speech Recognition
• Thank you for Attention: A survey on Attention-based Artificial Neural Networks for Automatic Speech Recognition
• Spike-Triggered Non-Autoregressive Transformer for End-to-End Speech Recognition
• SAN-M: Memory Equipped Self-Attention for End-to-End Speech Recognition
• Attention-based Transducer for Online Speech Recognition
• Improving RNN transducer with normalized jointer network
• Synchronous Transformers for End-to-End Speech Recognition
• Transformer with Bidirectional Decoder for Speech Recognition
• A Transformer with Interleaved Self-attention and Convolution for Hybrid Acoustic Models
• Rnn-transducer with language bias for end-to-end Mandarin-English code-switching speech recognition
• Transformer-based Online Speech Recognition with Decoder-end Adaptive Computation Steps
• Gated Recurrent Fusion with Joint Training Framework for Robust End-to-End Speech Recognition
• Adversarial Joint Training with Self-Attention Mechanism for Robust End-to-End Speech Recognition
• Conv-Transformer Transducer: Low Latency, Low Frame Rate, Streamable End-to-End Speech Recognition
• Self-and-Mixed Attention Decoder with Deep Acoustic Structure for Transformer-based LVCSR
• Unidirectional Memory-Self-Attention Transducer for Online Speech Recognition
• Multi-QuartzNet: Multi-Resolution Convolution for Speech Recognition with Multi-Layer Feature Fusion
• FSR: Accelerating the Inference Process of Transducer-Based Models by Applying Fast-Skip Regularization
• Context-Aware Transformer Transducer for Speech Recognition
• Multi-Channel Transformer Transducer for Speech Recognition
• Exploration of Audio Quality Assessment and Anomaly Localisation Using Attention Models
• Conformer-based End-to-end Speech Recognition With Rotary Position Embedding
• Wav-BERT: Cooperative Acoustic and Linguistic Representation Learning for Low-Resource Speech Recognition