Early learning of the optimal constant solution in neural networks and humans

Jirko Rubruck, Jan P. Bauer, Andrew Saxe +1

Deep neural networks learn increasingly complex functions over the course of training. Here, we show both empirically and theoretically that learning of the target function is prec…