On the Universality of Transformer Architectures; How Much Attention Is Enough?

Amirreza Abbasi, Mohsen Hooshmand

Transformers are crucial across many AI fields, such as large language models, computer vision, and reinforcement learning. This prominence stems from the architecture's perceived…