14 citations · 63 across the 15 of their papers we have counts for
16 papers
Benign Overfitting and the Geometry of the Ridge Regression Solution in Binary Classification
Alexander Tsigler, Luiz F. O. Chamon, Spencer Frei +1
In this work, we investigate the behavior of ridge regression in an overparameterized binary classification task. We assume examples are drawn from (anisotropic) class-conditional…
The Effect of SGD Batch Size on Autoencoder Learning: Sparsity, Sharpness, and Feature Learning
Nikhil Ghosh, Spencer Frei, Wooseok Ha +1
In this work, we investigate the dynamics of stochastic gradient descent (SGD) when training a single-neuron autoencoder with linear or ReLU activation on orthogonal data. We show…
Trained Transformers Learn Linear Models In-Context
Ruiqi Zhang, Spencer Frei, Peter L. Bartlett
Attention-based neural networks such as transformers have demonstrated a remarkable ability to exhibit in-context learning (ICL): Given a short prompt sequence of tokens from an un…
Benign Overfitting in Linear Classifiers and Leaky ReLU Networks from KKT Conditions for Margin Maximization
Spencer Frei, Gal Vardi, Peter L. Bartlett +1
Linear classifiers and leaky ReLU networks trained by gradient flow on the logistic loss have an implicit bias towards solutions which satisfy the Karush--Kuhn--Tucker (KKT) condit…
The Double-Edged Sword of Implicit Bias: Generalization vs. Robustness in ReLU Networks
Spencer Frei, Gal Vardi, Peter L. Bartlett +1
In this work, we study the implications of the implicit bias of gradient flow on generalization and adversarial robustness in ReLU networks. We focus on a setting where the data co…
Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data
Spencer Frei, Gal Vardi, Peter L. Bartlett +2
The implicit biases of gradient-based optimization algorithms are conjectured to be a major factor in the success of modern deep learning. In this work, we investigate the implicit…
Random Feature Amplification: Feature Learning and Generalization in Neural Networks
Spencer Frei, Niladri S. Chatterji, Peter L. Bartlett
In this work, we provide a characterization of the feature-learning process in two-layer ReLU networks trained by gradient descent on the logistic loss following random initializat…
Benign Overfitting without Linearity: Neural Network Classifiers Trained by Gradient Descent for Noisy Linear Data
Spencer Frei, Niladri S. Chatterji, Peter L. Bartlett
Benign overfitting, the phenomenon where interpolating models generalize well in the presence of noisy data, was first observed in neural network models trained with gradient desce…
Self-training Converts Weak Learners to Strong Learners in Mixture Models
Spencer Frei, Difan Zou, Zixiang Chen +1
We consider a binary classification problem when the data comes from a mixture of two rotationally symmetric distributions satisfying concentration and anti-concentration propertie…
Proxy Convexity: A Unified Framework for the Analysis of Neural Networks Trained by Gradient Descent
Spencer Frei, Quanquan Gu
Although the optimization objectives for learning neural networks are highly non-convex, gradient-based methods have been wildly successful at learning neural networks in practice.…
Provable Robustness of Adversarial Training for Learning Halfspaces with Noise
Difan Zou, Spencer Frei, Quanquan Gu
We analyze the properties of adversarial training for learning adversarially robust halfspaces in the presence of agnostic label noise. Denoting as the best ro…
Provable Generalization of SGD-trained Neural Networks of Any Width in the Presence of Adversarial Label Noise
Spencer Frei, Yuan Cao, Quanquan Gu
We consider a one-hidden-layer leaky ReLU network of arbitrary width trained by stochastic gradient descent (SGD) following an arbitrary initialization. We prove that SGD produces…
Agnostic Learning of Halfspaces with Gradient Descent via Soft Margins
Spencer Frei, Yuan Cao, Quanquan Gu
We analyze the properties of gradient descent on convex surrogates for the zero-one loss for the agnostic learning of linear halfspaces. If is the best classificatio…
Agnostic Learning of a Single Neuron with Gradient Descent
Spencer Frei, Yuan Cao, Quanquan Gu
We consider the problem of learning the best-fitting single neuron as measured by the expected square loss over some unknown…
Algorithm-Dependent Generalization Bounds for Overparameterized Deep Residual Networks
Spencer Frei, Yuan Cao, Quanquan Gu
The skip-connections used in residual networks have become a standard architecture choice in deep learning due to the increased training stability and generalization performance wi…
A lower bound for in range- bond percolation in two and three dimensions
Spencer Frei, Edwin Perkins
We use the connection between bond percolation and SIR epidemics to establish lower bounds for the critical percolation probability in and dimensions as the range becomes l…