#residual scaling

DeepLoop: Depth Scaling for Looped Transformers

Shuzhen Li, Yifan Zhang, Jiacheng Guo +2

DeepLoop reuses a compact stack of transformer blocks across multiple passes to increase model depth without adding parameters, and introduces new residual scaling rules to keep tr…