51 citations · 72 across the 7 of their papers we have counts for
4 papers · 1 filter
A Gentle Introduction to Gradient-Based Optimization and Variational Inequalities for Machine Learning
Neha S. Wadia, Yatin Dandi, Michael I. Jordan
The rapid progress in machine learning in recent years has been based on a highly productive connection to gradient-based optimization. Further progress hinges in part on a shift i…
Whitening and second order optimization both make information in the dataset unusable during training, and can reduce or prevent generalization
Neha S. Wadia, Daniel Duckworth, Samuel S. Schoenholz +2
Machine learning is predicated on the concept of generalization: a model achieving low error on a sufficiently large training set should also perform well on novel samples from the…
Critical Point-Finding Methods Reveal Gradient-Flat Regions of Deep Network Losses
Charles G. Frye, James Simon, Neha S. Wadia +3
Despite the fact that the loss functions of deep neural networks are highly non-convex, gradient-based optimization algorithms converge to approximately the same performance from m…
Numerically Recovering the Critical Points of a Deep Linear Autoencoder
Charles G. Frye, Neha S. Wadia, Michael R. DeWeese +1
Numerically locating the critical points of non-convex surfaces is a long-standing problem central to many fields. Recently, the loss surfaces of deep neural networks have been exp…