Thank you for Attention: A survey on Attention-based Artificial Neural Networks for Automatic Speech Recognition

Abstract

Attention is a very popular and effective mechanism in artificial neural network-based sequence-to-sequence models. In this survey paper, a comprehensive review of the different attention models used in developing automatic speech recognition systems is provided. The paper focuses on the development and evolution of attention models for offline and streaming speech recognition within recurrent neural network- and Transformer- based architectures.

Submitted to IEEE/ACM Trans. on Audio, Speech, and Language Processing

References in corpus (11)

• Attention-Based Models for Speech Recognition
• Sequence Transduction with Recurrent Neural Networks
• Voice Recognition Algorithms using Mel Frequency Cepstral Coefficient (MFCC) and Dynamic Time Warping (DTW) Techniques
• End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
• Self-Attention Transducers for End-to-End Speech Recognition
• Transformer-Transducer: End-to-End Speech Recognition with Self-Attention
• Online Automatic Speech Recognition with Listen, Attend and Spell Model
• Multi-head Monotonic Chunkwise Attention For Online Speech Recognition
• Low-Latency Sequence-to-Sequence Speech Recognition and Translation by Partial Hypothesis Selection
• Attention-based Transducer for Online Speech Recognition
• Weak-Attention Suppression For Transformer Based Speech Recognition