60 citations · 202 across the 22 of their papers we have counts for
22 papers
Approximation Rates for Shallow ReLU Neural Networks on Sobolev Spaces via the Radon Transform
Tong Mao, Jonathan W. Siegel, Jinchao Xu
Let be a bounded domain. We consider the problem of how efficiently shallow neural networks with the ReLU activation function can approximate functions…
Weighted variation spaces and approximation by shallow ReLU networks
Ronald DeVore, Robert D. Nowak, Rahul Parhi +1
We investigate the approximation of functions on a bounded domain by the outputs of single-hidden-layer ReLU neural networks of width . This form of…
Optimal Approximation of Zonoids and Uniform Approximation by Shallow Neural Networks
Jonathan W. Siegel
We study the following two related problems. The first is to determine to what error an arbitrary zonoid in can be approximated in the Hausdorff distance by a su…
Sharp Convergence Rates for Matching Pursuit
Jason M. Klusowski, Jonathan W. Siegel
We study the fundamental limits of matching pursuit, or the pure greedy algorithm, for approximating a target function by a linear combination of elements from a di…
Entropy-based convergence rates of greedy algorithms
Yuwen Li, Jonathan Siegel
We present convergence estimates of two types of greedy algorithms in terms of the metric entropy of underlying compact sets. In the first part, we measure the error of a standard…
Nesterov acceleration despite very noisy gradients
Kanan Gupta, Jonathan W. Siegel, Stephan Wojtowytsch
We present a generalization of Nesterov's accelerated gradient descent algorithm. Our algorithm (AGNES) provably achieves acceleration for smooth convex and strongly convex minimiz…
Sharp Lower Bounds on Interpolation by Deep ReLU Neural Networks at Irregularly Spaced Data
Jonathan W. Siegel
We study the interpolation power of deep ReLU neural networks. Specifically, we consider the question of how efficiently, in terms of the number of parameters, deep ReLU networks c…
Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spaces
Jonathan W. Siegel
Let be the unit cube in . We study the problem of how efficiently, in terms of the number of parameters, deep neural networks with the ReLU activation fu…
On the Activation Function Dependence of the Spectral Bias of Neural Networks
Qingguo Hong, Jonathan W. Siegel, Qinyang Tan +1
Neural networks are universal function approximators which are known to generalize well despite being dramatically overparameterized. We study this phenomenon from the point of vie…
Greedy Training Algorithms for Neural Networks and Applications to PDEs
Jonathan W. Siegel, Qingguo Hong, Xianlin Jin +2
Recently, neural networks have been widely applied for solving partial differential equations (PDEs). Although such methods have been proven remarkably successful on practical engi…
Characterization of the Variation Spaces Corresponding to Shallow Neural Networks
Jonathan W. Siegel, Jinchao Xu
We study the variation space corresponding to a dictionary of functions in for a bounded domain . Specifically, we compare the variation space, whic…
Optimal Convergence Rates for the Orthogonal Greedy Algorithm
Jonathan W. Siegel, Jinchao Xu
We analyze the orthogonal greedy algorithm when applied to dictionaries whose convex hull has small entropy. We show that if the metric entropy of the convex hull of $…
Sharp Lower Bounds on the Approximation Rate of Shallow Neural Networks
Jonathan W. Siegel, Jinchao Xu
We consider the approximation rates of shallow neural networks with respect to the variation norm. Upper bounds on these rates have been established for sigmoidal and ReLU activati…
A Priori Analysis of Stable Neural Network Solutions to Numerical PDEs
Qingguo Hong, Jonathan W. Siegel, Jinchao Xu
Methods for solving PDEs using neural networks have recently become a very important topic. We provide an a priori error analysis for such methods which is based on the $\mathcal{K…
Sharp Bounds on the Approximation Rates, Metric Entropy, and -widths of Shallow Neural Networks
Jonathan W. Siegel, Jinchao Xu
In this article, we study approximation properties of the variation spaces corresponding to shallow neural networks with a variety of activation functions. We introduce two main to…
High-Order Approximation Rates for Shallow Neural Networks with Cosine and ReLU Activation Functions
Jonathan W. Siegel, Jinchao Xu
We study the approximation properties of shallow neural networks with an activation function which is a power of the rectified linear unit. Specifically, we consider the dependence…
Extensible Structure-Informed Prediction of Formation Energy with Improved Accuracy and Usability employing Neural Networks
Adam M. Krajewski, Jonathan W. Siegel, Jinchao Xu +1
In the present paper, we introduce a new neural network-based tool for the prediction of formation energies of atomic structures based on elemental and structural features of Voron…
Training Sparse Neural Networks using Compressed Sensing
Jonathan W. Siegel, Jianhong Chen, Pengchuan Zhang +1
Pruning the weights of neural networks is an effective and widely-used technique for reducing model size and inference complexity. We develop and test a novel method based on compr…
Extended Regularized Dual Averaging Methods for Stochastic Optimization
Jonathan W. Siegel, Jinchao Xu
We introduce a new algorithm, extended regularized dual averaging (XRDA), for solving regularized stochastic optimization problems, which generalizes the regularized dual averaging…
Approximation Rates for Neural Networks with General Activation Functions
Jonathan W. Siegel, Jinchao Xu
We prove some new results concerning the approximation rate of neural networks with general activation functions. Our first result concerns the rate of approximation of a two layer…
Accelerated First-Order Methods: Differential Equations and Lyapunov Functions
Jonathan W. Siegel
We develop a theory of accelerated first-order optimization from the viewpoint of differential equations and Lyapunov functions. Building upon the previous work of many researchers…
Accelerated Optimization With Orthogonality Constraints
Jonathan W. Siegel
We develop a generalization of Nesterov's accelerated gradient descent method which is designed to deal with orthogonality constraints. To demonstrate the effectiveness of our meth…