Fixed Encoder Self-Attention Patterns in Transformer-Based Machine Translation

Alessandro Raganato, Yves Scherrer, Jörg Tiedemann

Transformer-based models have brought a radical change to neural machine translation. A key feature of the Transformer architecture is the so-called multi-head attention mechanism,…