Thank you for Attention: A survey on Attention-based Artificial Neural Networks for Automatic Speech Recognition

Abstract

Attention is a very popular and effective mechanism in artificial neural network-based sequence-to-sequence models. In this survey paper, a comprehensive review of the different attention models used in developing automatic speech recognition systems is provided. The paper focuses on the development and evolution of attention models for offline and streaming speech recognition within recurrent neural network- and Transformer- based architectures.

Submitted to IEEE/ACM Trans. on Audio, Speech, and Language Processing

References in corpus (9)

• Attention-Based Models for Speech Recognition
• Sequence Transduction with Recurrent Neural Networks
• Voice Recognition Algorithms using Mel Frequency Cepstral Coefficient (MFCC) and Dynamic Time Warping (DTW) Techniques
• End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
• Self-Attention Transducers for End-to-End Speech Recognition
• Transformer-Transducer: End-to-End Speech Recognition with Self-Attention
• Multi-head Monotonic Chunkwise Attention For Online Speech Recognition
• Attention-based Transducer for Online Speech Recognition
• Weak-Attention Suppression For Transformer Based Speech Recognition