Analysis of mean-field models arising from self-attention dynamics in transformer architectures with layer normalization

Martin Burger, Samira Kabri, Yury Korolev +2

The aim of this paper is to provide a mathematical analysis of transformer architectures using a self-attention mechanism with layer normalization. In particular, observed patterns…