3 papers
cs.LG2026
Enjoy Your Layer Normalization with the Computational Efficiency of RMSNorm
Yuxin Guo, Yihao Yue, Yunhao Ni +4
Layer normalization (LN) is a fundamental component in modern deep learning, but its per-sample centering and scaling introduce non-negligible inference overhead. RMSNorm improves…
cs.LG2025
Parallel Layer Normalization for Universal Approximation
Yunhao Ni, Yuxin Guo, Yuhe Liu +4
Layer Normalization (LN) is usually studied for its role in optimization, but it also acts as a multivariate nonlinearity between affine layers. We investigate how grouping LN unit…
cs.LG2024
On the Nonlinearity of Layer Normalization
Yunhao Ni, Yuxin Guo, Junlong Jia +1
Layer normalization (LN) is a ubiquitous technique in deep learning but our theoretical understanding to it remains elusive. This paper investigates a new theoretical direction for…