A Two-Phase Perspective on Deep Learning Dynamics

Robert de Mello Koch, Animik Ghosh

We propose that learning in deep neural networks proceeds in two phases: a rapid curve fitting phase followed by a slower compression or coarse graining phase. This view is support…