Rethinking Normalization Placement for LLMs: Post-Norm under Curriculum Depth Growing

Sheng Ren, Yadong Wang, Naiqiang Tan +7

Pre-norm is the standard normalization placement in modern Transformers because it facilitates joint optimization of full-depth models. We ask whether this preference persists when…