A Function-Space Approach to the Statistical Mechanics of Learning Dynamics

Yizhou Zhang, Weichen Wu, Lun Du +1

Deep neural networks exhibit regular macroscopic behavior despite highly nonlinear dynamics in vast parameter spaces. We develop a statistical-mechanical description of learning di…