14 citations · 75 across the 28 of their papers we have counts for
4 papers · 1 filter
Collegial Ensembles
Etai Littwin, Ben Myara, Sima Sabah +3
Modern neural network performance typically improves as model size increases. A recent line of research on the Neural Tangent Kernel (NTK) of over-parameterized networks indicates…
On Infinite-Width Hypernetworks
Etai Littwin, Tomer Galanti, Lior Wolf +1
{\em Hypernetworks} are architectures that produce the weights of a task-specific {\em primary network}. A notable application of hypernetworks in the recent literature involves le…
On the Convex Behavior of Deep Neural Networks in Relation to the Layers' Width
Etai Littwin, Lior Wolf
The Hessian of neural networks can be decomposed into a sum of two matrices: (i) the positive semidefinite generalized Gauss-Newton matrix G, and (ii) the matrix H containing negat…
On Random Kernels of Residual Architectures
Etai Littwin, Tomer Galanti, Lior Wolf
We derive finite width and depth corrections for the Neural Tangent Kernel (NTK) of ResNets and DenseNets. Our analysis reveals that finite size residual architectures are initiali…