102 citations · 157 across the 8 of their papers we have counts for
5 papers · 1 filter
Quantitative CLTs in Deep Neural Networks
Stefano Favaro, Boris Hanin, Domenico Marinucci +2
We study the distribution of a fully connected neural network with random Gaussian weights and biases in which the hidden layer widths are proportional to a large constant . Und…
Depth Dependence of P Learning Rates in ReLU MLPs
Samy Jelassi, Boris Hanin, Ziwei Ji +3
In this short note we consider random fully connected ReLU networks of width and depth equipped with a mean-field weight initialization. Our purpose is to study the depende…
How Data Augmentation affects Optimization for Linear Regression
Boris Hanin, Yi Sun
Though data augmentation has rapidly emerged as a key tool for optimization in modern machine learning, a clear picture of how augmentation schedules affect optimization and intera…
Finite Depth and Width Corrections to the Neural Tangent Kernel
Boris Hanin, Mihai Nica
We prove the precise scaling, at finite depth and width, for the mean and variance of the neural tangent kernel (NTK) in a randomly initialized ReLU network. The standard deviation…
Nonlinear Approximation and (Deep) ReLU Networks
I. Daubechies, R. DeVore, S. Foucart +2
This article is concerned with the approximation and expressive powers of deep neural networks. This is an active research area currently producing many interesting papers. The res…