290 citations · 701 across the 29 of their papers we have counts for
4 papers · 1 filter
Exploring the Uncertainty Properties of Neural Networks' Implicit Priors in the Infinite-Width Limit
Ben Adlam, Jaehoon Lee, Lechao Xiao +2
Modern deep learning models have achieved great success in predictive accuracy for many data modalities. However, their application to many real-world tasks is restricted by poor u…
Finite Versus Infinite Neural Networks: an Empirical Study
Jaehoon Lee, Samuel S. Schoenholz, Jeffrey Pennington +4
We perform a careful, thorough, and large scale empirical study of the correspondence between wide neural networks and kernel methods. By doing so, we resolve a variety of open que…
The Surprising Simplicity of the Early-Time Learning Dynamics of Neural Networks
Wei Hu, Lechao Xiao, Ben Adlam +1
Modern neural networks are often regarded as complex black-box functions whose behavior is difficult to understand owing to their nonlinear dependence on the data and the nonconvex…
Provable Benefit of Orthogonal Initialization in Optimizing Deep Linear Networks
Wei Hu, Lechao Xiao, Jeffrey Pennington
The selection of initial parameter values for gradient-based optimization of deep neural networks is one of the most impactful hyperparameter choices in deep learning systems, affe…