Demystifying Lazy Training of Neural Networks from a Macroscopic Viewpoint

Yuqing Li, Tao Luo, Qixuan Zhou

In this paper, we advance the understanding of neural network training dynamics by examining the intricate interplay of various factors introduced by weight parameters in the initi…