Deep Networks Learn Deep Hierarchical Models

Amit Daniely

We consider supervised learning with $n$ labels and show that layerwise SGD on residual networks can efficiently learn a class of hierarchical models. This model class assumes the…