60 citations · 202 across the 22 of their papers we have counts for
4 papers · 1 filter
Extended Regularized Dual Averaging Methods for Stochastic Optimization
Jonathan W. Siegel, Jinchao Xu
We introduce a new algorithm, extended regularized dual averaging (XRDA), for solving regularized stochastic optimization problems, which generalizes the regularized dual averaging…
Approximation Rates for Neural Networks with General Activation Functions
Jonathan W. Siegel, Jinchao Xu
We prove some new results concerning the approximation rate of neural networks with general activation functions. Our first result concerns the rate of approximation of a two layer…
Accelerated First-Order Methods: Differential Equations and Lyapunov Functions
Jonathan W. Siegel
We develop a theory of accelerated first-order optimization from the viewpoint of differential equations and Lyapunov functions. Building upon the previous work of many researchers…
Accelerated Optimization With Orthogonality Constraints
Jonathan W. Siegel
We develop a generalization of Nesterov's accelerated gradient descent method which is designed to deal with orthogonality constraints. To demonstrate the effectiveness of our meth…