4 papers · 1 filter
There Will Be a Scientific Theory of Deep Learning
Jamie Simon, Daniel Kunin, Alexander Atanasov +11
In this paper, we make the case that a scientific theory of deep learning is emerging. By this we mean a theory which characterizes important properties and statistics of the train…
Deep Learning as a Convex Paradigm of Computation: Minimizing Circuit Size with ResNets
Arthur Jacot
This paper argues that DNNs implement a computational Occam's razor -- finding the `simplest' algorithm that fits the data -- and that this could explain their incredible and wide-…
Hamiltonian Mechanics of Feature Learning: Bottleneck Structure in Leaky ResNets
Arthur Jacot, Alexandre Kaiser
We study Leaky ResNets, which interpolate between ResNets and Fully-Connected nets depending on an 'effective depth' hyper-parameter . In the infinite depth limit, we st…
How DNNs break the Curse of Dimensionality: Compositionality and Symmetry Learning
Arthur Jacot, Seok Hoan Choi, Yuxiao Wen
We show that deep neural networks (DNNs) can efficiently learn any composition of functions with bounded -norm, which allows DNNs to break the curse of dimensionality in way…