A Geometric Theory for Nonequilibrium Transformer Dynamics

Aythami Bethencourt de León, Darryl D. Holm

We develop a variational and geometric theory based on statistical physics for the joint evolution of Transformer attention and token representations at fixed model parameters. To…