75 citations · 131 across the 20 of their papers we have counts for
15 papers · 1 filter
Convergence of Steepest Descent and Adam under Non-Uniform Smoothness
Sharan Vaswani, Yifan Sun, Reza Babanezhad
Recent work has analyzed the convergence of first-order methods under non-uniform smoothness assumptions that better model the loss landscape in machine learning tasks. We generali…
Towards Parameter-Free Temporal Difference Learning
Yunxiang Li, Mark Schmidt, Reza Babanezhad +1
Temporal difference (TD) learning is a fundamental algorithm for estimating value functions in reinforcement learning. Recent finite-time analyses of TD with linear function approx…
Dissecting Discrete Soft Actor-Critic: Limitations and Principled Alternatives
Reza Asad, Reza Babanezhad, Sharan Vaswani
While Soft Actor-Critic (SAC) is highly effective in continuous control, its discrete counterpart (DSAC) performs poorly on challenging discrete-action domains such as Atari. Conse…
Armijo Line-search Can Make (Stochastic) Gradient Descent Provably Faster
Sharan Vaswani, Reza Babanezhad
Armijo line-search (Armijo-LS) is a standard method to set the step-size for gradient descent (GD). For smooth functions, Armijo-LS alleviates the need to know the global smoothnes…
Fast Convergence of Softmax Policy Mirror Ascent
Reza Asad, Reza Babanezhad, Issam Laradji +2
Natural policy gradient (NPG) is a common policy optimization algorithm and can be viewed as mirror ascent in the space of probabilities. Recently, Vaswani et al. [2021] introduced…
Fast Online Node Labeling for Very Large Graphs
Baojian Zhou, Yifan Sun, Reza Babanezhad
This paper studies the online node classification problem under a transductive learning setting. Current methods either invert a graph kernel matrix with runtime…
Decision-Aware Actor-Critic with Function Approximation and Theoretical Guarantees
Sharan Vaswani, Amirreza Kazemi, Reza Babanezhad +1
Actor-critic (AC) methods are widely used in reinforcement learning (RL) and benefit from the flexibility of using any policy gradient method as the actor and value-based method as…
Target-based Surrogates for Stochastic Optimization
Jonathan Wilder Lavington, Sharan Vaswani, Reza Babanezhad +2
We consider minimizing functions for which it is expensive to compute the (possibly stochastic) gradient. Such functions are prevalent in reinforcement learning, imitation learning…
Towards Painless Policy Optimization for Constrained MDPs
Arushi Jain, Sharan Vaswani, Reza Babanezhad +2
We study policy optimization in an infinite horizon, -discounted constrained Markov decision process (CMDP). Our objective is to return a policy that achieves large expected rew…
SVRG Meets AdaGrad: Painless Variance Reduction
Benjamin Dubois-Taine, Sharan Vaswani, Reza Babanezhad +2
Variance reduction (VR) methods for finite-sum minimization typically require the knowledge of problem-dependent constants that are often unknown and difficult to estimate. To addr…
Geometry-Aware Universal Mirror-Prox
Reza Babanezhad, Simon Lacoste-Julien
Mirror-prox (MP) is a well-known algorithm to solve variational inequality (VI) problems. VI with a monotone operator covers a large group of settings such as convex minimization,…
To Each Optimizer a Norm, To Each Norm its Generalization
Sharan Vaswani, Reza Babanezhad, Jose Gallego-Posada +3
We study the implicit regularization of optimization methods for linear models interpolating the training data in the under-parametrized and over-parametrized regimes. Since it is…
Reducing the variance in online optimization by transporting past gradients
Sébastien M. R. Arnold, Pierre-Antoine Manzagol, Reza Babanezhad +2
Most stochastic optimization methods use gradients once before discarding them. While variance reduction methods have shown that reusing past gradients can be beneficial when there…
M-ADDA: Unsupervised Domain Adaptation with Deep Metric Learning
Issam Laradji, Reza Babanezhad
Unsupervised domain adaptation techniques have been successful for a wide range of problems where supervised labels are limited. The task is to classify an unlabeled `target' datas…
Stop Wasting My Gradients: Practical SVRG
Reza Babanezhad, Mohamed Osama Ahmed, Alim Virani +3
We present and analyze several strategies for improving the performance of stochastic variance-reduced gradient (SVRG) methods. We first show that the convergence rate of these met…