23 citations · 53 across the 11 of their papers we have counts for
13 papers
Provably-Efficient and Internally-Deterministic Parallel Union-Find
Alexander Fedorov, Diba Hashemi, Giorgi Nadiradze +1
Determining the degree of inherent parallelism in classical sequential algorithms and leveraging it for fast parallel execution is a key topic in parallel computing, and detailed a…
Hybrid Decentralized Optimization: Leveraging Both First- and Zeroth-Order Optimizers for Faster Convergence
Matin Ansaripour, Shayan Talaei, Giorgi Nadiradze +1
Distributed optimization is the standard way of speeding up machine learning training, and most of the research in the area focuses on distributed first-order, gradient-based metho…
Communication-Efficient Federated Learning With Data and Client Heterogeneity
Hossein Zakerinia, Shayan Talaei, Giorgi Nadiradze +1
Federated Learning (FL) enables large-scale distributed training of machine learning models, while still allowing individual nodes to maintain data locally. However, executing FL a…
Multi-Queues Can Be State-of-the-Art Priority Schedulers
Anastasiia Postnikova, Nikita Koval, Giorgi Nadiradze +1
Designing and implementing efficient parallel priority schedulers is an active research area. An intriguing proposed design is the Multi-Queue: given threads and disti…
Lower Bounds for Shared-Memory Leader Election under Bounded Write Contention
Dan Alistarh, Rati Gelashvili, Giorgi Nadiradze
This paper gives tight logarithmic lower bounds on the solo step complexity of leader election in an asynchronous shared-memory model with single-writer multi-reader (SWMR) registe…
Breaking (Global) Barriers in Parallel Stochastic Optimization with Wait-Avoiding Group Averaging
Shigang Li, Tal Ben-Nun, Giorgi Nadiradze +4
Deep learning at scale is dominated by communication time. Distributing samples across nodes usually yields the best performance, but poses scaling challenges due to global informa…
Efficiency Guarantees for Parallel Incremental Algorithms under Relaxed Schedulers
Dan Alistarh, Nikita Koval, Giorgi Nadiradze
Several classic problems in graph processing and computational geometry are solved via incremental algorithms, which split computation into a series of small tasks acting on shared…
Dynamic Averaging Load Balancing on Cycles
Dan Alistarh, Giorgi Nadiradze, Amirmojtaba Sabour
We consider the following dynamic load-balancing process: given an underlying graph with nodes, in each step , one unit of load is created, and placed at a randoml…
Asynchronous Decentralized SGD with Quantized and Local Updates
Giorgi Nadiradze, Amirmojtaba Sabour, Peter Davies +2
Decentralized optimization is emerging as a viable alternative for scalable distributed machine learning, but also introduces new challenges in terms of synchronization costs. To t…
Relaxed Schedulers Can Efficiently Parallelize Iterative Algorithms
Dan Alistarh, Trevor Brown, Justin Kopinsky +1
There has been significant progress in understanding the parallelism inherent to iterative sequential algorithms: for many classic algorithms, the depth of the dependence structure…
The Transactional Conflict Problem
Dan Alistarh, Syed Kamran Haider, Raphael Kübler +1
The transactional conflict problem arises in transactional systems whenever two or more concurrent transactions clash on a data item. While the standard solution to such conflicts…
Distributionally Linearizable Data Structures
Dan Alistarh, Trevor Brown, Justin Kopinsky +2
Relaxed concurrent data structures have become increasingly popular, due to their scalability in graph processing and machine learning applications. Despite considerable interest,…
The Power of Choice in Priority Scheduling
Dan Alistarh, Justin Kopinsky, Jerry Li +1
Consider the following random process: we are given queues, into which elements of increasing labels are inserted uniformly at random. To remove an element, we pick two queues…