239 citations · 1.1k across the 35 of their papers we have counts for
42 papers
The Score Hamiltonian: Mapping Diffusion Models to Adiabatic Transport
Peter Halmos, Boris Hanin
We exhibit an exact correspondence between sampling with score-based diffusion models and adiabatic transport of ground states for a family of Schrödinger operators we call Score H…
Top Singular Value in Sum-Products of Random Matrices
Kevin Han Huang, Boris Hanin
We study the top singular value for a sum of independent random matrices, each of which is a product of i.i.d. Gaussian matrices. Our main conceptu…
Global Universality of Singular Values in Products of Many Large Random Matrices
Boris Hanin, Tianze Jiang
We study the singular values (and Lyapunov exponents) for products of independent random matrices with i.i.d. entries. Such matrix products have been extensively an…
Quantitative CLTs in Deep Neural Networks
Stefano Favaro, Boris Hanin, Domenico Marinucci +2
We study the distribution of a fully connected neural network with random Gaussian weights and biases in which the hidden layer widths are proportional to a large constant . Und…
Principles for Initialization and Architecture Selection in Graph Neural Networks with ReLU Activations
Gage DeZoort, Boris Hanin
This article derives and validates three principles for initialization and architecture selection in finite width graph neural networks (GNNs) with ReLU activations. First, we theo…
Depth Dependence of P Learning Rates in ReLU MLPs
Samy Jelassi, Boris Hanin, Ziwei Ji +3
In this short note we consider random fully connected ReLU networks of width and depth equipped with a mean-field weight initialization. Our purpose is to study the depende…
Bayesian Interpolation with Deep Linear Networks
Boris Hanin, Alexander Zlokapa
Characterizing how neural network depth, width, and dataset size jointly impact model quality is a central problem in deep learning theory. We give here a complete solution in the…
Maximal Initial Learning Rates in Deep ReLU Networks
Gaurav Iyer, Boris Hanin, David Rolnick
Training a neural network requires choosing a suitable learning rate, which involves a trade-off between speed and effectiveness of convergence. While there has been considerable t…
Scaling asymptotics of spectral Wigner functions
Boris Hanin, Steve Zelditch
We prove that smooth Wigner-Weyl spectral sums at an energy level exhibit Airy scaling asymptotics across the classical energy surface . This was proved earlier by the aut…
Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis
Wuyang Chen, Wei Huang, Xinyu Gong +2
Advanced deep neural networks (DNNs), designed by either human or AutoML algorithms, are growing increasingly complex. Diverse operations are connected by complicated connectivity…
Random Fully Connected Neural Networks as Perturbatively Solvable Hierarchies
Boris Hanin
This article considers fully connected neural networks with Gaussian random weights and biases as well as hidden layers, each of width proportional to a large parameter . Fo…
Ridgeless Interpolation with Shallow ReLU Networks in is Nearest Neighbor Curvature Extrapolation and Provably Generalizes on Lipschitz Functions
Boris Hanin
We prove a precise geometric description of all one layer ReLU networks with a single linear unit and input/output dimensions equal to one that interpolate a given dataset…
Random Neural Networks in the Infinite Width Limit as Gaussian Processes
Boris Hanin
This article gives a new proof that fully connected neural networks with random weights and biases converge to Gaussian processes in the regime where the input dimension, output di…
The Principles of Deep Learning Theory
Daniel A. Roberts, Sho Yaida, Boris Hanin
This book develops an effective theory approach to understanding deep neural networks of practical relevance. Beginning from a first-principles component-level picture of networks,…
Deep ReLU Networks Preserve Expected Length
Boris Hanin, Ryan Jeong, David Rolnick
Assessing the complexity of functions computed by a neural network helps us understand how the network will learn and generalize. One natural measure of complexity is how the netwo…
Neural Network Approximation
Ronald DeVore, Boris Hanin, Guergana Petrova
Neural Networks (NNs) are the method of choice for building learning algorithms. Their popularity stems from their empirical success on several challenging learning problems. Howev…
How Data Augmentation affects Optimization for Linear Regression
Boris Hanin, Yi Sun
Though data augmentation has rapidly emerged as a key tool for optimization in modern machine learning, a clear picture of how augmentation schedules affect optimization and intera…
Non-asymptotic Results for Singular Values of Gaussian Matrix Products
Boris Hanin, Grigoris Paouris
This article concerns the non-asymptotic analysis of the singular values (and Lyapunov exponents) of Gaussian matrix products in the regime where the number of term in the pro…
Finite Depth and Width Corrections to the Neural Tangent Kernel
Boris Hanin, Mihai Nica
We prove the precise scaling, at finite depth and width, for the mean and variance of the neural tangent kernel (NTK) in a randomly initialized ReLU network. The standard deviation…
Deep ReLU Networks Have Surprisingly Few Activation Patterns
Boris Hanin, David Rolnick
The success of deep networks has been attributed in part to their expressivity: per parameter, deep networks can approximate a richer class of functions than shallow networks. In R…
Nonlinear Approximation and (Deep) ReLU Networks
I. Daubechies, R. DeVore, S. Foucart +2
This article is concerned with the approximation and expressive powers of deep neural networks. This is an active research area currently producing many interesting papers. The res…
Interface Asymptotics of Wigner-Weyl Distributions for the Harmonic Oscillator
Boris Hanin, Steve Zelditch
We prove several types of scaling results for Wigner distributions of spectral projections of the isotropic Harmonic oscillator on . In prior work, we studied Wigner d…
Interface Asymptotics of Eigenspace Wigner distributions for the Harmonic Oscillator
Boris Hanin, Steve Zelditch
Eigenspaces of the quantum isotropic Harmonic Oscillator on have extremally high multiplicites and the…
Complexity of Linear Regions in Deep Networks
Boris Hanin, David Rolnick
It is well-known that the expressivity of a neural network depends on its architecture, with deeper networks expressing more complex functions. In the case of networks that compute…