2 papers
math.OC2026
A First-Order Mean Field Control Analysis of Transformer Layers under Cross-Entropy Training
Cheng Huan, Hongwei Yuan
We study Transformer-type residual layers under cross-entropy training through a continuous-depth mean field control viewpoint. Depth is treated as time, layer parameters as contro…
math.OC2026
A Mean-Field Analysis of Multi-Head Self-Attention under Cross-Entropy Training
Cheng Huan, Hongfwei Yuan
This paper develops a mean-field theory for a simplified single-layer causal multi-head self-attention model trained by cross-entropy minimization. Each attention head is treated a…