Expanding Expressivity in Transformer Models with MöbiusAttention

Anna-Maria Halacheva, Mojtaba Nayyeri, Steffen Staab

Attention mechanisms and Transformer architectures have revolutionized Natural Language Processing (NLP) by enabling exceptional modeling of long-range dependencies and capturing i…