123 citations · 711 across the 24 of their papers we have counts for
24 papers
The Implicit Regularization of Dynamical Stability in Stochastic Gradient Descent
Lei Wu, Weijie J. Su
In this paper, we study the implicit regularization of stochastic gradient descent (SGD) through the lens of {\em dynamical stability} (Wu et al., 2018). We start by revising exist…
Embedding Inequalities for Barron-type Spaces
Lei Wu
An important problem in machine learning theory is to understand the approximation and generalization properties of two-layer neural networks in high dimensions. To this end, resea…
A duality framework for analyzing random feature and two-layer neural networks
Hongrui Chen, Jihao Long, Lei Wu
We consider the problem of learning functions within the and Barron spaces, which play crucial roles in understanding random feature models (RFMs), two-layer ne…
The alignment property of SGD noise and how it helps select flat minima: A stability analysis
Lei Wu, Mingze Wang, Weijie Su
The phenomenon that stochastic gradient descent (SGD) favors flat minima has played a critical role in understanding the implicit regularization of SGD. In this paper, we provide a…
Beyond the Quadratic Approximation: the Multiscale Structure of Neural Network Loss Landscapes
Chao Ma, Daniel Kunin, Lei Wu +1
A quadratic approximation of neural network loss landscapes has been extensively used to study the optimization process of these networks. Though, it usually holds in a very small…
Learning a Single Neuron for Non-monotonic Activation Functions
Lei Wu
We study the problem of learning a single neuron with gradient descent (GD). All the existing positive results are limited to the case…