3 papers
math.OC2024
Second-order Information Promotes Mini-Batch Robustness in Variance-Reduced Gradients
Sachin Garg, Albert S. Berahas, Michał Dereziński
We show that, for finite-sum minimization problems, incorporating partial second-order information of the objective function can dramatically improve the robustness to mini-batch s…
math.OC2023
Non-Uniform Smoothness for Gradient Descent
Albert S. Berahas, Lindon Roberts, Fred Roosta
The analysis of gradient descent-type methods typically relies on the Lipschitz continuity of the objective gradient. This generally requires an expensive hyperparameter tuning pro…
math.OC2023
Adaptive Consensus: A network pruning approach for decentralized optimization
Suhail M. Shah, Albert S. Berahas, Raghu Bollapragada
We consider network-based decentralized optimization problems, where each node in the network possesses a local function and the objective is to collectively attain a consensus sol…