Showing 2026Show all
2 papers · 1 filter
cs.LG2026
SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers
Shaowen Wang, Ge Zhang, Kairong Luo +6
Looped Transformers increase effective depth by iterating a shared block of layers, but most evaluations compare at fixed model size, conflating architectural advantage with extra…
cs.LG2026
Loop-P: Residual Scaling and Hyperparameter Transfer in Looped Transformers
Shaowen Wang, Bingrui Li, Ge Zhang +4
Looped Transformers have drawn growing attention because looping improves multi-step reasoning and increases the executed depth of a model without adding parameters. They achieve t…