3 papers
cs.LG2026
Anti Mode-Collapse in Mean-Field Transformer via Auxiliary Variables
Masaaki Imaizumi, Masanori Koyama, Noboru Isobe +1
We use a mean-field-based transformer model to theoretically investigate how auxiliary variables, such as positional encoding, prevent mode collapse of self-attention mechanisms. T…
cs.LG2026
Training-Induced Escape from Token Clustering in a Mean-Field Formulation of Transformers
Noboru Isobe, Daisuke Inoue, Masaaki Imaizumi
Transformers perform inference by iteratively transforming token representations across layers. This layerwise computation has been studied empirically, and recent mean-field theor…
cs.LG2024
Flow matching achieves almost minimax optimal convergence
Kenji Fukumizu, Taiji Suzuki, Noboru Isobe +2
Flow matching (FM) has gained significant attention as a simulation-free generative model. Unlike diffusion models, which are based on stochastic differential equations, FM employs…