2 citations · 3 across the 6 of their papers we have counts for
7 papers
ACCO: Accumulate While You Communicate for Communication-Overlapped Sharded LLM Training
Adel Nabli, Louis Fournier, Pierre Erbacher +3
Training LLMs relies on distributed implementations using multiple GPUs to compute gradients in parallel with sharded optimizers. However, synchronizing gradients in data parallel…
WASH: Train your Ensemble with Communication-Efficient Weight Shuffling, then Average
Louis Fournier, Adel Nabli, Masih Aminbeidokhti +3
The performance of deep neural networks is enhanced by ensemble methods, which average the output of several models. However, this comes at an increased cost at inference. Weight a…
: Accelerating Asynchronous Communication in Decentralized Deep Learning
Adel Nabli, Eugene Belilovsky, Edouard Oyallon
Distributed training of Deep Learning models has been critical to many recent successes in the field. Current standard methods primarily rely on synchronous centralized algorithms…
DADAO: Decoupled Accelerated Decentralized Asynchronous Optimization
Adel Nabli, Edouard Oyallon
This work introduces DADAO: the first decentralized, accelerated, asynchronous, primal, first-order algorithm to minimize a sum of -smooth and -strongly convex functions dist…
Speech Sequence Embeddings using Nearest Neighbors Contrastive Learning
Robin Algayres, Adel Nabli, Benoit Sagot +1
We introduce a simple neural encoder architecture that can be trained using an unsupervised contrastive learning objective which gets its positive samples from data-augmented k-Nea…
Curriculum learning for multilevel budgeted combinatorial problems
Adel Nabli, Margarida Carvalho
Learning heuristics for combinatorial optimization problems through graph neural networks have recently shown promising results on some classic NP-hard problems. These are single-l…
Complexity of the Multilevel Critical Node Problem
Adel Nabli, Margarida Carvalho, Pierre Hosteins
In this work, we analyze a sequential game played in a graph called the Multilevel Critical Node problem (MCN). A defender and an attacker are the players of this game. The defende…