217 citations · 1.4k across the 33 of their papers we have counts for
6 papers · 1 filter
The Eigenlearning Framework: A Conservation Law Perspective on Kernel Regression and Wide Neural Networks
James B. Simon, Madeline Dickens, Dhruva Karkada +1
We derive simple closed-form estimates for the test risk and other generalization metrics of kernel ridge regression (KRR). Relative to prior work, our derivations are greatly simp…
Reverse Engineering the Neural Tangent Kernel
James B. Simon, Sajant Anand, Michael R. DeWeese
The development of methods to guide the design of neural networks is an important open challenge for deep learning theory. As a paradigm for principled neural architecture design,…
A new method for parameter estimation in probabilistic models: Minimum probability flow
Jascha Sohl-Dickstein, Peter Battaglino, Michael R. DeWeese
Fitting probabilistic models to data is often difficult, due to the general intractability of the partition function. We propose a new parameter fitting method, Minimum Probability…
Critical Point-Finding Methods Reveal Gradient-Flat Regions of Deep Network Losses
Charles G. Frye, James Simon, Neha S. Wadia +3
Despite the fact that the loss functions of deep neural networks are highly non-convex, gradient-based optimization algorithms converge to approximately the same performance from m…
Numerically Recovering the Critical Points of a Deep Linear Autoencoder
Charles G. Frye, Neha S. Wadia, Michael R. DeWeese +1
Numerically locating the critical points of non-convex surfaces is a long-standing problem central to many fields. Recently, the loss surfaces of deep neural networks have been exp…
Minimum Probability Flow Learning
Jascha Sohl-Dickstein, Peter Battaglino, Michael R. DeWeese
Fitting probabilistic models to data is often difficult, due to the general intractability of the partition function and its derivatives. Here we propose a new parameter estimation…