On Difficulties of Attention Factorization through Shared Memory

Uladzislau Yorsh, Martin Holeňa, Ondřej Bojar +1

Transformers have revolutionized deep learning in numerous fields, including natural language processing, computer vision, and audio processing. Their strength lies in their attent…