5 papers
Local MixVR: Breaking the Communication-Sample Dependence in Distributed Learning
Tehila Dahan, Bassel Hamoud, Roie Reshef +2
Communication overhead is a crucial bottleneck in scalable distributed learning. While existing methods aim to efficiently utilize data points, such as Local SGD, Minibatch SGD, an…
Bringing Order to Asynchronous SGD: Towards Optimality under Data-Dependent Delays with Momentum
Tehila Dahan, Roie Reshef, Sharon Goldstein +1
Asynchronous stochastic gradient descent (SGD) enables scalable distributed training but suffers from gradient staleness. Existing mitigation strategies, such as delay-adaptive lea…
Optimal Sample Complexity for Single Time-Scale Actor-Critic with Momentum
Navdeep Kumar, Tehila Dahan, Lior Cohen +4
We establish an optimal sample complexity of for obtaining an -optimal global policy using a single-timescale actor-critic (AC) algorithm in infinite-horizon disco…
Weight for Robustness: A Comprehensive Approach towards Optimal Fault-Tolerant Asynchronous ML
Tehila Dahan, Kfir Y. Levy
We address the challenges of Byzantine-robust training in asynchronous distributed machine learning systems, aiming to enhance efficiency amid massive parallelization and heterogen…
-SGD: Stable Stochastic Optimization via a Double Momentum Mechanism
Tehila Dahan, Kfir Y. Levy
We consider stochastic convex optimization problems where the objective is an expectation over smooth functions. For this setting we suggest a novel gradient estimate that combines…