3 papers
stat.ML2026
An Adjoint-Sensitivity Framework for Lost-in-the-Middle Phenomena in Causal Residual Transformers
Cheng Huan, Hongwei Yuan
We develop an adjoint-sensitivity framework for positional influence in causal residual Transformers and separate unconditional analytic results from conditional boundary-shape con…
math.OC2026
A First-Order Mean Field Control Analysis of Transformer Layers under Cross-Entropy Training
Cheng Huan, Hongwei Yuan
We study Transformer-type residual layers under cross-entropy training through a continuous-depth mean field control viewpoint. Depth is treated as time, layer parameters as contro…
math.OC2026
A Mean-Field Analysis of Multi-Head Self-Attention under Cross-Entropy Training
Cheng Huan, Hongfwei Yuan
This paper develops a mean-field theory for a simplified single-layer causal multi-head self-attention model trained by cross-entropy minimization. Each attention head is treated a…