Weight Initialization and Variance Dynamics in Deep Neural Networks and Large Language Models

Yankun Han

Weight initialization governs signal propagation and gradient flow at the start of training. This paper offers a theory-grounded and empirically validated study across two regimes:…