A Function-Space Approach to the Statistical Mechanics of Learning Dynamics

Yizhou Zhang, Weichen Wu, Lun Du +1

In the kernel regime, neural-network learning inherits its preferences from a frozen spectrum. During feature learning, this spectrum evolves, yet networks retain systematic biases…