1 citations · 1 across the 5 of their papers we have counts for
Showing 2026Show all
2 papers · 1 filter
cs.LG2026
Anti Mode-Collapse in Mean-Field Transformer via Auxiliary Variables
Masaaki Imaizumi, Masanori Koyama, Noboru Isobe +1
We use a mean-field-based transformer model to theoretically investigate how auxiliary variables, such as positional encoding, prevent mode collapse of self-attention mechanisms. T…
cs.LG2026
Training-Induced Escape from Token Clustering in a Mean-Field Formulation of Transformers
Noboru Isobe, Daisuke Inoue, Masaaki Imaizumi
Transformers perform inference by iteratively transforming token representations across layers. This layerwise computation has been studied empirically, and recent mean-field theor…