Attention as a Guide for Simultaneous Speech Translation
arXiv:2212.07850 · doi:10.18653/v1/2023.acl-long.745
Abstract
The study of the attention mechanism has sparked interest in many fields, such as language modeling and machine translation. Although its patterns have been exploited to perform different tasks, from neural network understanding to textual alignment, no previous work has analysed the encoder-decoder attention behavior in speech translation (ST) nor used it to improve ST on a specific task. In this paper, we fill this gap by proposing an attention-based policy (EDAtt) for simultaneous ST (SimulST) that is motivated by an analysis of the existing attention relations between audio input and textual output. Its goal is to leverage the encoder-decoder attention scores to guide inference in real time. Results on en->{de, es} show that the EDAtt policy achieves overall better results compared to the SimulST state of the art, especially in terms of computational-aware latency.
Accepted to ACL 2023
References in corpus (13)
- Sequence Transduction with Recurrent Neural Networks
- fairseq S2T: Fast Speech-to-Text Modeling with fairseq
- Do Attention Heads in BERT Track Syntactic Dependencies?
- Adding Interpretable Attention to Neural Translation Models Improves Word Alignment
- Squeezeformer: An Efficient Transformer for Automatic Speech Recognition
- SimulMT to SimulST: Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translation
- Over-Generation Cannot Be Rewarded: Length-Adaptive Average Lagging for Simultaneous Speech Translation
- Efficient yet Competitive Speech Translation: FBK@IWSLT2022
- Exploring Continuous Integrate-and-Fire for Adaptive Simultaneous Speech Translation
- Does Simultaneous Speech Translation need Simultaneous Models?
- Pay Better Attention to Attention: Head Selection in Multilingual and Multi-Domain Sequence Modeling
- Decision Attentive Regularization to Improve Simultaneous Speech Translation Systems
- When Good and Reproducible Results are a Giant with Feet of Clay: The Importance of Software Quality in NLP