7 citations · 7 across the 2 of their papers we have counts for
5 papers · 1 filter
Escaping Saddle Points Faster with Stochastic Momentum
Jun-Kun Wang, Chi-Heng Lin, Jacob Abernethy
Stochastic gradient descent (SGD) with stochastic momentum is popular in nonconvex stochastic optimization and particularly for the training of deep neural networks. In standard SG…
Understanding How Over-Parametrization Leads to Acceleration: A case of learning a single teacher neuron
Jun-Kun Wang, Jacob Abernethy
Over-parametrization has become a popular technique in deep learning. It is observed that by over-parametrization, a larger neural network needs a fewer training iterations than a…
Revisiting Projection-Free Optimization for Strongly Convex Constraint Sets
Jarrid Rector-Brooks, Jun-Kun Wang, Barzan Mozafari
We revisit the Frank-Wolfe (FW) optimization under strongly convex constraint sets. We provide a faster convergence rate for FW without line search, showing that a previously overl…
Acceleration through Optimistic No-Regret Dynamics
Jun-Kun Wang, Jacob Abernethy
We consider the problem of minimizing a smooth convex function by reducing the optimization to computing the Nash equilibrium of a particular zero-sum convex-concave game. Zero-sum…
Faster Rates for Convex-Concave Games
Jacob Abernethy, Kevin A. Lai, Kfir Y. Levy +1
We consider the use of no-regret algorithms to compute equilibria for particular classes of convex-concave games. While standard regret bounds would lead to convergence rates on th…