50 citations · 193 across the 27 of their papers we have counts for
6 papers · 1 filter
Nesterov acceleration despite very noisy gradients
Kanan Gupta, Jonathan W. Siegel, Stephan Wojtowytsch
We present a generalization of Nesterov's accelerated gradient descent algorithm. Our algorithm (AGNES) provably achieves acceleration for smooth convex and strongly convex minimiz…
Optimal bump functions for shallow ReLU networks: Weight decay, depth separation and the curse of dimensionality
Stephan Wojtowytsch
In this note, we study how neural networks with a single hidden layer and ReLU activation interpolate data drawn from a radially symmetric distribution with target labels 1 at the…
Stochastic gradient descent with noise of machine learning type. Part I: Discrete time analysis
Stephan Wojtowytsch
Stochastic gradient descent (SGD) is one of the most popular algorithms in modern machine learning. The noise encountered in these applications is different from that in many theor…
A priori estimates for classification problems using neural networks
Weinan E, Stephan Wojtowytsch
We consider binary and multi-class classification problems using hypothesis classes of neural networks. For a given hypothesis class, we use Rademacher complexity estimates and dir…
On the Banach spaces associated with multi-layer ReLU networks: Function representation, approximation theory and gradient descent dynamics
Weinan E, Stephan Wojtowytsch
We develop Banach spaces for ReLU neural networks of finite depth and infinite width. The spaces contain all finite fully connected -layer networks and their -limiting…
Representation formulas and pointwise properties for Barron functions
Weinan E, Stephan Wojtowytsch
We study the natural function space for infinitely wide two-layer neural networks with ReLU activation (Barron space) and establish different representation formulae. In two cases,…