33 citations · 133 across the 43 of their papers we have counts for
5 papers · 1 filter
The extremal landscape for the CE ensemble
Elliot Paquette, Ofer Zeitouni
We consider the extremes of the logarithm of the characteristic polynomial of matrices from the CE ensemble. We prove convergence in distribution of the centered maxima (of the…
Implicit Regularization or Implicit Conditioning? Exact Risk Trajectories of SGD in High Dimensions
Courtney Paquette, Elliot Paquette, Ben Adlam +1
Stochastic gradient descent (SGD) is a pillar of modern machine learning, serving as the go-to optimization algorithm for a diverse array of problems. While the empirical success o…
Trajectory of Mini-Batch Momentum: Batch Size Saturation and Convergence in High Dimensions
Kiwon Lee, Andrew N. Cheng, Courtney Paquette +1
We analyze the dynamics of large batch stochastic gradient descent with momentum (SGD+M) on the least squares problem when both the number of samples and dimensions are large. In t…
Flatness of the nuclear norm sphere, simultaneous polarization, and uniqueness in nuclear norm minimization
Tim Hoheisel, Elliot Paquette
In this paper we establish necessary and sufficient conditions for the existence of line segments (or flats) in the sphere of the nuclear norm via the notion of simultaneous polari…
Homogenization of SGD in high-dimensions: Exact dynamics and generalization properties
Courtney Paquette, Elliot Paquette, Ben Adlam +1
We develop a stochastic differential equation, called homogenized SGD, for analyzing the dynamics of stochastic gradient descent (SGD) on a high-dimensional random least squares pr…