565 citations · 1k across the 21 of their papers we have counts for
5 papers · 1 filter
Gradients are Not All You Need
Luke Metz, C. Daniel Freeman, Samuel S. Schoenholz +1
Differentiable programming techniques are widely used in the community and are responsible for the machine learning renaissance of the past several decades. While these methods are…
Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping
James Martens, Andy Ballard, Guillaume Desjardins +4
Using an extended and formalized version of the Q/C map analysis of Poole et al. (2016), along with Neural Tangent Kernel theory, we identify the main pathologies present in deep n…
Learn2Hop: Learned Optimization on Rough Landscapes
Amil Merchant, Luke Metz, Sam Schoenholz +1
Optimization of non-convex loss surfaces containing many local minima remains a critical problem in a variety of domains, including operations research, informatics, and material d…
dPV: An End-to-End Differentiable Solar-Cell Simulator
Sean Mann, Eric Fadel, Samuel S. Schoenholz +3
We introduce dPV, an end-to-end differentiable photovoltaic (PV) cell simulator based on the drift-diffusion model and Beer-Lambert law for optical absorption. dPV is programmed in…
Tilting the playing field: Dynamical loss functions for machine learning
Miguel Ruiz-Garcia, Ge Zhang, Samuel S. Schoenholz +1
We show that learning can be improved by using loss functions that evolve cyclically during training to emphasize one class at a time. In underparameterized networks, such dynamica…