89 citations · 285 across the 12 of their papers we have counts for
20 papers
Gradient Descent and the Power Method: Exploiting their connection to find the leftmost eigen-pair and escape saddle points
Rachael Tappenden, Martin Takáč
This work shows that applying Gradient Descent (GD) with a fixed step size to minimize a (possibly nonconvex) quadratic function is equivalent to running the Power Method (PM) on t…
FLECS-CGD: A Federated Learning Second-Order Framework via Compression and Sketching with Compressed Gradient Differences
Artem Agafonov, Brahim Erraji, Martin Takáč
In the recent paper FLECS (Agafonov et al, FLECS: A Federated Learning Second-Order Framework via Compression and Sketching), the second-order framework FLECS was proposed for the…
Robustness Analysis of Classification Using Recurrent Neural Networks with Perturbed Sequential Input
Guangyi Liu, Arash Amini, Martin Takac +1
For a given stable recurrent neural network (RNN) that is trained to perform a classification task using sequential inputs, we quantify explicit robustness bounds as a function of…
Distributed Learning With Sparsified Gradient Differences
Yicheng Chen, Rick S. Blum, Martin Takac +1
A very large number of communications are typically required to solve distributed learning tasks, and this critically limits scalability and convergence speed in wireless communica…
Don't Forget Your Teacher: A Corrective Reinforcement Learning Framework
Mohammadreza Nazari, Majid Jahani, Lawrence V. Snyder +1
Although reinforcement learning (RL) can provide reliable solutions in many settings, practitioners are often wary of the discrepancies between the RL solution and their status quo…
New Convergence Aspects of Stochastic Gradient Algorithms
Lam M. Nguyen, Phuong Ha Nguyen, Peter Richtárik +3
The classical convergence analysis of SGD is carried out under the assumption that the norm of the stochastic gradient is uniformly bounded. While this might hold for some loss fun…