28 citations · 52 across the 11 of their papers we have counts for
15 papers
Gradient Descent and the Power Method: Exploiting their connection to find the leftmost eigen-pair and escape saddle points
Rachael Tappenden, Martin Takáč
This work shows that applying Gradient Descent (GD) with a fixed step size to minimize a (possibly nonconvex) quadratic function is equivalent to running the Power Method (PM) on t…
FLECS: A Federated Learning Second-Order Framework via Compression and Sketching
Artem Agafonov, Dmitry Kamzolov, Rachael Tappenden +2
Inspired by the recent work FedNL (Safaryan et al, FedNL: Making Newton-Type Methods Applicable to Federated Learning), we propose a new communication efficient second-order framew…
Stochastic Gradient Methods with Preconditioned Updates
Abdurakhmon Sadiev, Aleksandr Beznosikov, Abdulla Jasem Almansoori +3
This work considers the non-convex finite sum minimization problem. There are several algorithms for such problems, but existing methods often work poorly when the problem is badly…
Decentralized Personalized Federated Learning: Lower Bounds and Optimal Algorithm for All Personalization Modes
Abdurakhmon Sadiev, Ekaterina Borodich, Aleksandr Beznosikov +5
This paper considers the problem of decentralized, personalized federated learning. For centralized personalized federated learning, a penalty that measures the deviation from the…
SONIA: A Symmetric Blockwise Truncated Optimization Algorithm
Majid Jahani, Mohammadreza Nazari, Rachael Tappenden +2
This work presents a new algorithm for empirical risk minimization. The algorithm bridges the gap between first- and second-order methods by computing a search direction that uses…
Gradient and Hessian approximations in Derivative Free Optimization
Ian D. Coope, Rachael Tappenden
This work investigates finite differences and the use of interpolation models to obtain approximations to the first and second derivatives of a function. Here, it is shown that if…