The Asymptotic Behavior of Attention in Transformers

Álvaro Rodríguez Abella, João Pedro Silvestre, Paulo Tabuada

The transformer architecture has become the foundation of modern Large Language Models (LLMs), yet its theoretical properties are still not well understood. As with classic neural…