4 citations · 16 across the 14 of their papers we have counts for
Showing 2026 · stat.MLShow all
2 papers · 2 filters
stat.ML2026
Gaussian Equivalence for Multi-Head Self-Attention
Tomohiro Hayase, Ryo Karakida
A theoretical understanding of multi-head self-attention is fundamental to the study of modern neural networks. Using random matrix theory, we establish Gaussian equivalence for mu…
stat.ML2026
A Unified Framework for Critical Scaling of Inverse Temperature in Self-Attention
Tomohiro Hayase, Ryo Karakida
Length-dependent logit rescaling is widely used to stabilize long-context self-attention, but existing analyses and methods suggest conflicting inverse-temperature laws for the con…