3 papers
cs.LG2026
Enjoy Your Layer Normalization with the Computational Efficiency of RMSNorm
Yuxin Guo, Yihao Yue, Yunhao Ni +4
Layer normalization (LN) is a fundamental component in modern deep learning, but its per-sample centering and scaling introduce non-negligible inference overhead. RMSNorm improves…
cs.LG2025
Parallel Layer Normalization for Universal Approximation
Yunhao Ni, Yuxin Guo, Yuhe Liu +4
This paper studies the approximation capabilities of neural networks that combine layer normalization (LN) with linear layers. We prove that networks consisting of two linear layer…
cs.LG2024
On the Nonlinearity of Layer Normalization
Yunhao Ni, Yuxin Guo, Junlong Jia +1
Layer normalization (LN) is a ubiquitous technique in deep learning but our theoretical understanding to it remains elusive. This paper investigates a new theoretical direction for…