3.8k citations · 7.6k across the 48 of their papers we have counts for
4 papers · 1 filter
What does a deep neural network confidently perceive? The effective dimension of high certainty class manifolds and their low confidence boundaries
Stanislav Fort, Ekin Dogus Cubuk, Surya Ganguli +1
Deep neural network classifiers partition input space into high confidence regions for each class. The geometry of these class manifolds (CMs) is widely studied and intimately rela…
Deep equilibrium networks are sensitive to initialization statistics
Atish Agarwala, Samuel S. Schoenholz
Deep equilibrium networks (DEQs) are a promising way to construct models which trade off memory for compute. However, theoretical understanding of these models is still lacking com…
Fast Finite Width Neural Tangent Kernel
Roman Novak, Jascha Sohl-Dickstein, Samuel S. Schoenholz
The Neural Tangent Kernel (NTK), defined as where $\left[\partia…
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao +448
Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabil…