908 citations · 2.4k across the 149 of their papers we have counts for
16 papers · 1 filter
Byzantine-Resilient Non-Convex Stochastic Gradient Descent
Zeyuan Allen-Zhu, Faeze Ebrahimian, Jerry Li +1
We study adversary-resilient stochastic distributed optimization, in which machines can independently compute stochastic gradients, and cooperate to jointly optimize over their…
Adaptive Gradient Quantization for Data-Parallel SGD
Fartash Faghri, Iman Tabrizian, Ilia Markov +3
Many communication-efficient variants of SGD use gradient quantization schemes. These schemes are often heuristic and fixed over the course of training. We empirically observe that…
Towards Tight Communication Lower Bounds for Distributed Optimisation
Dan Alistarh, Janne H. Korhonen
We consider a standard distributed optimisation setting where machines, each holding a -dimensional function , aim to jointly minimise the sum of the functions $\sum_{i…
The Splay-List: A Distribution-Adaptive Concurrent Skip-List
Vitaly Aksenov, Dan Alistarh, Alexandra Drozdova +1
The design and implementation of efficient concurrent data structures have seen significant attention. However, most of this work has focused on concurrent data structures providin…
Fast General Distributed Transactions with Opacity using Global Time
Alex Shamis, Matthew Renzelmann, Stanko Novakovic +6
Transactions can simplify distributed applications by hiding data distribution, concurrency, and failures from the application developer. Ideally the developer would see the abstra…
Stochastic Gradient Langevin with Delayed Gradients
Vyacheslav Kungurtsev, Bapi Chatterjee, Dan Alistarh
Stochastic Gradient Langevin Dynamics (SGLD) ensures strong guarantees with regards to convergence in measure for sampling log-concave posterior distributions by adding noise to st…