Quantifying Concentration Phenomena of Mean-Field Transformers in the Low-Temperature Regime

Albert Alcalde, Leon Bungert, Konstantin Riedl +1

Transformers with self-attention modules as their core components have become an integral architecture in modern large language and foundation models. In this paper, we study the e…