6 papers
Local MixVR: Breaking the Communication-Sample Dependence in Distributed Learning
Tehila Dahan, Bassel Hamoud, Roie Reshef +2
Communication overhead is a crucial bottleneck in scalable distributed learning. While existing methods aim to efficiently utilize data points, such as Local SGD, Minibatch SGD, an…
Bringing Order to Asynchronous SGD: Towards Optimality under Data-Dependent Delays with Momentum
Tehila Dahan, Roie Reshef, Sharon Goldstein +1
Asynchronous SGD enables scalable distributed training but suffers from stale gradients. When delays depend on the data, slower samples may be underrepresented, biasing training to…
Optimal Sample Complexity for Single Time-Scale Actor-Critic with Momentum
Navdeep Kumar, Tehila Dahan, Lior Cohen +4
We establish an optimal sample complexity of for obtaining an -optimal global policy using a single-timescale actor-critic (AC) algorithm in infinite-horizon discoun…
Weight for Robustness: A Comprehensive Approach towards Optimal Fault-Tolerant Asynchronous ML
Tehila Dahan, Kfir Y. Levy
We address the challenges of Byzantine-robust training in asynchronous distributed machine learning systems, aiming to enhance efficiency amid massive parallelization and heterogen…
Fault Tolerant ML: Efficient Meta-Aggregation and Synchronous Training
Tehila Dahan, Kfir Y. Levy
In this paper, we investigate the challenging framework of Byzantine-robust training in distributed machine learning (ML) systems, focusing on enhancing both efficiency and practic…
-SGD: Stable Stochastic Optimization via a Double Momentum Mechanism
Tehila Dahan, Kfir Y. Levy
We consider stochastic convex optimization problems where the objective is an expectation over smooth functions. For this setting we suggest a novel gradient estimate that combines…