2 citations · 2 across the 3 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.LG2024
On Multi-Stage Loss Dynamics in Neural Networks: Mechanisms of Plateau and Descent Stages
Zheng-An Chen, Tao Luo, GuiHong Wang
The multi-stage phenomenon in the training loss curves of neural networks has been widely observed, reflecting the non-linearity and complexity inherent in the training process. In…
cs.LG2024★ 2 cited
On the dynamics of three-layer neural networks: initial condensation
Zheng-An Chen, Tao Luo
Empirical and theoretical works show that the input weights of two-layer neural networks, when initialized with small values, converge towards isolated orientations. This phenomeno…