743 citations · 2.1k across the 27 of their papers we have counts for
5 papers · 1 filter
A Differentiable Rank-Based Objective For Better Feature Learning
Krunoslav Lehman Pavasovic, David Lopez-Paz, Giulio Biroli +1
In this paper, we leverage existing statistical methods to better understand feature learning from data. We tackle this by modifying the model-free variable selection method, Featu…
Measuring and signing fairness as performance under multiple stakeholder distributions
David Lopez-Paz, Diane Bouchacourt, Levent Sagun +1
As learning machines increase their influence on decisions concerning human lives, analyzing their fairness properties becomes a subject of central importance. Yet, our best tools…
On the Heavy-Tailed Theory of Stochastic Gradient Descent for Deep Neural Networks
Umut Şimşekli, Mert Gürbüzbalaban, Thanh Huy Nguyen +2
The gradient noise (GN) in the stochastic gradient descent (SGD) algorithm is often considered to be Gaussian in the large data regime by assuming that the \emph{classical} central…
Comparing Dynamics: Deep Neural Networks versus Glassy Systems
M. Baity-Jesi, L. Sagun, M. Geiger +6
We analyze numerically the training dynamics of deep neural networks (DNN) by using methods developed in statistical physics of glassy systems. The two main issues we address are (…
Explorations on high dimensional landscapes
Levent Sagun, V. Ugur Guney, Gerard Ben Arous +1
Finding minima of a real valued non-convex function over a high dimensional space is a major challenge in science. We provide evidence that some such functions that are defined on…