50 citations · 193 across the 27 of their papers we have counts for
32 papers
The Barron-Lipschitz Energy Gap and Depth Separation Phenomena in Scientific Machine Learning
Nima Rezaei, Stephan Wojtowytsch
We illustrate in several examples that even neural networks of infinite width (specifically, Barron functions) may encounter substantial obstacles when used as a model class for pr…
Convex-concave splitting for the Allen-Cahn equation leads to -slow movement of interfaces
Patrick Dondl, Akwum Onwunta, Ludwig Striet +1
The convex-concave splitting discretization of the Allen-Cahn is easy to implement and guaranteed to be energy decreasing even for large time-steps. We analyze the time-stepping sc…
Momentum-based minimization of the Ginzburg-Landau functional on Euclidean spaces and graphs
Oluwatosin Akande, Patrick Dondl, Kanan Gupta +2
We study the momentum-based minimization of a diffuse perimeter functional on Euclidean spaces and on graphs with applications to semi-supervised classification tasks in machine le…
Nesterov acceleration in benignly non-convex landscapes
Kanan Gupta, Stephan Wojtowytsch
While momentum-based optimization algorithms are commonly used in the notoriously non-convex optimization problems of deep learning, their analysis has historically been restricted…
A note on spatially inhomogeneous Cahn-Hilliard energies
Stephan Wojtowytsch
In 2023, Cristoferi, Fonseca and Ganedi proved that Cahn-Hilliard type energies with spatially inhomogeneous potentials converge to the usual (isotropic and homogeneous) perimeter…
SineNet: Learning Temporal Dynamics in Time-Dependent Partial Differential Equations
Xuan Zhang, Jacob Helwig, Yuchao Lin +4
We consider using deep neural networks to solve time-dependent partial differential equations (PDEs), where multi-scale processing is crucial for modeling complex, time-evolving dy…
Group Equivariant Fourier Neural Operators for Partial Differential Equations
Jacob Helwig, Xuan Zhang, Cong Fu +3
We consider solving partial differential equations (PDEs) with Fourier neural operators (FNOs), which operate in the frequency domain. Since the laws of physics do not depend on th…
Nesterov acceleration despite very noisy gradients
Kanan Gupta, Jonathan W. Siegel, Stephan Wojtowytsch
We present a generalization of Nesterov's accelerated gradient descent algorithm. Our algorithm (AGNES) provably achieves acceleration for smooth convex and strongly convex minimiz…
Optimal bump functions for shallow ReLU networks: Weight decay, depth separation and the curse of dimensionality
Stephan Wojtowytsch
In this note, we study how neural networks with a single hidden layer and ReLU activation interpolate data drawn from a radially symmetric distribution with target labels 1 at the…
Qualitative neural network approximation over R and C: Elementary proofs for analytic and polynomial activation
Josiah Park, Stephan Wojtowytsch
In this article, we prove approximation theorems in classes of deep and shallow neural networks with analytic activation functions by elementary arguments. We prove for both real a…
Stochastic gradient descent with noise of machine learning type. Part II: Continuous time analysis
Stephan Wojtowytsch
The representation of functions by artificial neural networks depends on a large number of parameters in a non-linear fashion. Suitable parameters of these are found by minimizing…
Stochastic gradient descent with noise of machine learning type. Part I: Discrete time analysis
Stephan Wojtowytsch
Stochastic gradient descent (SGD) is one of the most popular algorithms in modern machine learning. The noise encountered in these applications is different from that in many theor…
On the emergence of simplex symmetry in the final and penultimate layers of neural network classifiers
Weinan E, Stephan Wojtowytsch
A recent numerical study observed that neural network classifiers enjoy a large degree of symmetry in the penultimate layer. Namely, if where is a linear map…
Some observations on high-dimensional partial differential equations with Barron data
Weinan E, Stephan Wojtowytsch
We use explicit representation formulas to show that solutions to certain partial differential equations lie in Barron spaces or multilayer spaces if the PDE data lie in such funct…
A priori estimates for classification problems using neural networks
Weinan E, Stephan Wojtowytsch
We consider binary and multi-class classification problems using hypothesis classes of neural networks. For a given hypothesis class, we use Rademacher complexity estimates and dir…
Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don't
Weinan E, Chao Ma, Stephan Wojtowytsch +1
The purpose of this article is to review the achievements made in the last few years towards the understanding of the reasons behind the success and subtleties of neural network-ba…
On the Banach spaces associated with multi-layer ReLU networks: Function representation, approximation theory and gradient descent dynamics
Weinan E, Stephan Wojtowytsch
We develop Banach spaces for ReLU neural networks of finite depth and infinite width. The spaces contain all finite fully connected -layer networks and their -limiting…
Connected Coulomb Columns: Analysis and Numerics
Patrick Dondl, Matteo Novaga, Stephan Wojtowytsch +1
We consider a version of Gamow's liquid drop model with a short range attractive perimeter-penalizing potential and a long-range Coulomb interaction of a uniformly charged mass in…
Representation formulas and pointwise properties for Barron functions
Weinan E, Stephan Wojtowytsch
We study the natural function space for infinitely wide two-layer neural networks with ReLU activation (Barron space) and establish different representation formulae. In two cases,…
On the Convergence of Gradient Descent Training for Two-layer ReLU-networks in the Mean Field Regime
Stephan Wojtowytsch
We describe a necessary and sufficient condition for the convergence to minimum Bayes risk when training two-layer ReLU-networks by gradient descent in the mean field regime with o…
Can Shallow Neural Networks Beat the Curse of Dimensionality? A mean field training perspective
Stephan Wojtowytsch, Weinan E
We prove that the gradient descent training of a two-layer neural network on empirical or population risk may not decrease population risk at an order faster than un…
Kolmogorov Width Decay and Poor Approximators in Machine Learning: Shallow Neural Networks, Random Feature Models and Neural Tangent Kernels
Weinan E, Stephan Wojtowytsch
We establish a scale separation of Kolmogorov width type between subspaces of a given Banach space under the condition that a sequence of linear maps converges much faster on one o…
On the motion of curved dislocations in three dimensions: Simplified linearized elasticity
Irene Fonseca, Janusz Ginster, Stephan Wojtowytsch
It is shown that in core-radius cutoff regularized simplified elasticity (where the elastic energy depends quadratically on the full displacement gradient rather than its symmetriz…
Confined elasticae and the buckling of cylindrical shells
Stephan Wojtowytsch
For curves of prescribed length embedded into the unit disc in two dimensions, we obtain scaling results for the minimal elastic energy as the length just exceeds and in the l…