Improving Transformers with Probabilistic Attention Keys

Tam Nguyen, Tan M. Nguyen, Dung D. Le +5

Multi-head attention is a driving force behind state-of-the-art transformers, which achieve remarkable performance across a variety of natural language processing (NLP) and compute…