A Unified Perspective on the Dynamics of Deep Transformers

Valérie Castin, Pierre Ablin, José Antonio Carrillo +1

Transformers, which are state-of-the-art in most machine learning tasks, represent the data as sequences of vectors called tokens. This representation is then exploited by the atte…