9 citations · 10 across the 2 of their papers we have counts for
6 papers
On the Activation Function Dependence of the Spectral Bias of Neural Networks
Qingguo Hong, Jonathan W. Siegel, Qinyang Tan +1
Neural networks are universal function approximators which are known to generalize well despite being dramatically overparameterized. We study this phenomenon from the point of vie…
Sharp Lower Bounds on the Approximation Rate of Shallow Neural Networks
Jonathan W. Siegel, Jinchao Xu
We consider the approximation rates of shallow neural networks with respect to the variation norm. Upper bounds on these rates have been established for sigmoidal and ReLU activati…
Training Sparse Neural Networks using Compressed Sensing
Jonathan W. Siegel, Jianhong Chen, Pengchuan Zhang +1
Pruning the weights of neural networks is an effective and widely-used technique for reducing model size and inference complexity. We develop and test a novel method based on compr…
Approximation Rates for Neural Networks with General Activation Functions
Jonathan W. Siegel, Jinchao Xu
We prove some new results concerning the approximation rate of neural networks with general activation functions. Our first result concerns the rate of approximation of a two layer…
Accelerated First-Order Methods: Differential Equations and Lyapunov Functions
Jonathan W. Siegel
We develop a theory of accelerated first-order optimization from the viewpoint of differential equations and Lyapunov functions. Building upon the previous work of many researchers…
Accelerated Optimization With Orthogonality Constraints
Jonathan W. Siegel
We develop a generalization of Nesterov's accelerated gradient descent method which is designed to deal with orthogonality constraints. To demonstrate the effectiveness of our meth…