33 citations · 133 across the 43 of their papers we have counts for
53 papers
Anisotropic local law for non-separable sample covariance matrices
Zhou Fan, Renyuan Ma, Elliot Paquette +1
We establish local laws for sample covariance matrices $K = N^{-1}\sum_{i=1}^N \g_i\g_i^*$ where the random vectors $\g_1, \ldots, \g_N \in \R^n$ are independent with common covari…
High-Dimensional Privacy-Utility Dynamics of Noisy Stochastic Gradient Descent on Least Squares
Shurong Lin, Eric D. Kolaczyk, Adam Smith +1
The interplay between optimization and privacy has become a central theme in privacy-preserving machine learning. Noisy stochastic gradient descent (SGD) has emerged as a cornersto…
Eigenvalue distribution of the Neural Tangent Kernel in the quadratic scaling
Lucas Benigni, Elliot Paquette
We compute the asymptotic eigenvalue distribution of the neural tangent kernel of a two-layer neural network under a specific scaling of dimension. Namely, if $X\in\mathbb{R}^{n\ti…
Bulk asymptotics of the Gaussian -ensemble characteristic polynomial
Gaultier Lambert, Elliot Paquette
The Gaussian -ensemble (GE) is a fundamental model in random matrix theory. In this paper, we provide a comprehensive asymptotic description of the characteristic polynomial…
The Fourier coefficients of the holomorphic multiplicative chaos in the limit of large frequency
Joseph Najnudel, Elliot Paquette, Nick Simm +1
The holomorphic multiplicative chaos (HMC) is a holomorphic analogue of the Gaussian multiplicative chaos. It arises naturally as the limit in large matrix size of the characterist…
Operator level soft edge to bulk transition in -ensembles via canonical systems
Vincent Painchaud, Elliot Paquette
The stochastic Airy and sine operators, which are respectively a random Sturm-Liouville operator and a random Dirac operator, characterize the soft edge and bulk scaling limits of…
The image of random analytic functions: coverage of the complex plane via branching processes
Alon Nishry, Elliot Paquette
We consider the range of random analytic functions with finite radius of convergence. We show that any unbounded random Taylor series with rotationally invariant coefficients has d…
Dyson Equation for Correlated Linearizations and Test Error of Random Features Regression
Hugo Latourelle-Vigeant, Elliot Paquette
This paper develops some theory of the Dyson equation for correlated linearizations and uses it to solve a problem on asymptotic deterministic equivalent for the test error in rand…
Hitting the High-Dimensional Notes: An ODE for SGD learning dynamics on GLMs and multi-index models
Elizabeth Collins-Woodfin, Courtney Paquette, Elliot Paquette +1
We analyze the dynamics of streaming stochastic gradient descent (SGD) in the high-dimensional limit when applied to generalized linear models and multi-index models (e.g. logistic…
Differentially Private Linear Regression with Linked Data
Shurong Lin, Elliot Paquette, Eric D. Kolaczyk
There has been increasing demand for establishing privacy-preserving methodologies for modern statistics and machine learning. Differential privacy, a mathematical notion from comp…
Fitting an ellipsoid to a quadratic number of random points
Afonso S. Bandeira, Antoine Maillard, Shahar Mendelson +1
We consider the problem of fitting standard Gaussian random vectors in to the boundary of a centered ellipsoid, as . This problem…
High-dimensional limit of one-pass SGD on least squares
Elizabeth Collins-Woodfin, Elliot Paquette
We give a description of the high-dimensional limit of one-pass single-batch stochastic gradient descent (SGD) on a least squares problem. This limit is taken with non-vanishing st…
The extremal landscape for the CE ensemble
Elliot Paquette, Ofer Zeitouni
We consider the extremes of the logarithm of the characteristic polynomial of matrices from the CE ensemble. We prove convergence in distribution of the centered maxima (of the…
Implicit Regularization or Implicit Conditioning? Exact Risk Trajectories of SGD in High Dimensions
Courtney Paquette, Elliot Paquette, Ben Adlam +1
Stochastic gradient descent (SGD) is a pillar of modern machine learning, serving as the go-to optimization algorithm for a diverse array of problems. While the empirical success o…
Trajectory of Mini-Batch Momentum: Batch Size Saturation and Convergence in High Dimensions
Kiwon Lee, Andrew N. Cheng, Courtney Paquette +1
We analyze the dynamics of large batch stochastic gradient descent with momentum (SGD+M) on the least squares problem when both the number of samples and dimensions are large. In t…
Flatness of the nuclear norm sphere, simultaneous polarization, and uniqueness in nuclear norm minimization
Tim Hoheisel, Elliot Paquette
In this paper we establish necessary and sufficient conditions for the existence of line segments (or flats) in the sphere of the nuclear norm via the notion of simultaneous polari…
Homogenization of SGD in high-dimensions: Exact dynamics and generalization properties
Courtney Paquette, Elliot Paquette, Ben Adlam +1
We develop a stochastic differential equation, called homogenized SGD, for analyzing the dynamics of stochastic gradient descent (SGD) on a high-dimensional random least squares pr…
Random geometric graphs and the spherical Wishart matrix
Elliot Paquette, Andrew Vander Werf
We consider the random geometric graph on vertices drawn uniformly from a --dimensional sphere. We focus on the sparse regime, when the expected degree is constant independe…
Dynamics of Stochastic Momentum Methods on Large-scale, Quadratic Models
Courtney Paquette, Elliot Paquette
We analyze a class of stochastic gradient algorithms with momentum on a high-dimensional random least squares problem. Our framework, inspired by random matrix theory, provides an…
SGD in the Large: Average-case Analysis, Asymptotics, and Stepsize Criticality
Courtney Paquette, Kiwon Lee, Fabian Pedregosa +1
We propose a new framework, inspired by random matrix theory, for analyzing the dynamics of stochastic gradient descent (SGD) when both number of samples and dimensions are large.…
Secular Coefficients and the Holomorphic Multiplicative Chaos
Joseph Najnudel, Elliot Paquette, Nick Simm
We study the secular coefficients of random unitary matrices drawn from the Circular -Ensemble, which are defined as the coefficients of in the ch…
Strong approximation of Gaussian -ensemble characteristic polynomials: the edge regime and the stochastic Airy function
Gaultier Lambert, Elliot Paquette
We investigate the characteristic polynomials of the Gaussian -ensemble for general through its transfer matrix recurrence. We show that the rescaled characteristic polyno…
Anchored expansion of Delaunay complexes in real hyperbolic space and stationary point processes
Itai Benjamini, Yoav Krauz, Elliot Paquette
We give sufficient conditions for a discrete set of points in any dimensional real hyperbolic space to have positive anchored expansion. The first condition is a bounded mean densi…
Universality for the conjugate gradient and MINRES algorithms on sample covariance matrices
Elliot Paquette, Thomas Trogdon
We present a probabilistic analysis of two Krylov subspace methods for solving linear systems. We prove a central limit theorem for norms of the residual vectors that are produced…