239 citations · 1.1k across the 35 of their papers we have counts for
5 papers · 1 filter
Bayesian Interpolation with Deep Linear Networks
Boris Hanin, Alexander Zlokapa
Characterizing how neural network depth, width, and dataset size jointly impact model quality is a central problem in deep learning theory. We give here a complete solution in the…
Maximal Initial Learning Rates in Deep ReLU Networks
Gaurav Iyer, Boris Hanin, David Rolnick
Training a neural network requires choosing a suitable learning rate, which involves a trade-off between speed and effectiveness of convergence. While there has been considerable t…
Scaling asymptotics of spectral Wigner functions
Boris Hanin, Steve Zelditch
We prove that smooth Wigner-Weyl spectral sums at an energy level exhibit Airy scaling asymptotics across the classical energy surface . This was proved earlier by the aut…
Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis
Wuyang Chen, Wei Huang, Xinyu Gong +2
Advanced deep neural networks (DNNs), designed by either human or AutoML algorithms, are growing increasingly complex. Diverse operations are connected by complicated connectivity…
Random Fully Connected Neural Networks as Perturbatively Solvable Hierarchies
Boris Hanin
This article considers fully connected neural networks with Gaussian random weights and biases as well as hidden layers, each of width proportional to a large parameter . Fo…