14 papers
Convergence of Steepest Descent and Adam under Non-Uniform Smoothness
Sharan Vaswani, Yifan Sun, Reza Babanezhad
Recent work has analyzed the convergence of first-order methods under non-uniform smoothness assumptions that better model the loss landscape in machine learning tasks. We generali…
Glocal Smoothness: Line search and adaptive step sizes can help in theory too!
Curtis Fox, Aaron Mishkin, Sharan Vaswani +1
Iteration complexities for optimizing smooth functions with first-order algorithms are typically stated in terms of a global Lipschitz constant of the gradient, and near-optimal re…
Augmented Lagrangian Method for Last-Iterate Convergence for Constrained MDPs
Michael Lu, Max Qiushi Lin, Mo Chen +1
We study policy optimization for infinite-horizon, discounted constrained Markov decision processes (CMDPs). While existing theoretical guarantees typically hold for the mixture po…
Dissecting Discrete Soft Actor-Critic: Limitations and Principled Alternatives
Reza Asad, Reza Babanezhad, Sharan Vaswani
While Soft Actor-Critic (SAC) is highly effective in continuous control, its discrete counterpart (DSAC) performs poorly on challenging discrete-action domains such as Atari. Conse…
Optimistic Actor-Critic with Parametric Policies for Linear Markov Decision Processes
Max Qiushi Lin, Reza Asad, Kevin Tan +3
Although actor-critic methods have been successful in practice, their theoretical analyses have several limitations. Specifically, existing theoretical work either sidesteps the ex…
Towards Noise-adaptive, Problem-adaptive (Accelerated) Stochastic Gradient Descent
Sharan Vaswani, Benjamin Dubois-Taine, Reza Babanezhad
We aim to make stochastic gradient descent (SGD) adaptive to (i) the noise in the stochastic gradients and (ii) problem-dependent constants. When minimizing smooth, strongly…