output
20162025
most citedRevisiting Fundamentals of Experience Replay

81 citations

researchers with a paper here
collaborating institutions
Showing 2024Show all

3 papers · 1 filter

cs.LG2024

ACCO: Accumulate While You Communicate for Communication-Overlapped Sharded LLM Training

Adel Nabli, Louis Fournier, Pierre Erbacher +3

Training LLMs relies on distributed implementations using multiple GPUs to compute gradients in parallel with sharded optimizers. However, synchronizing gradients in data parallel…

cs.LG2024

PETRA: Parallel End-to-end Training with Reversible Architectures

Stéphane Rivaud, Louis Fournier, Thomas Pumir +3

Reversible architectures have been shown to be capable of performing on par with their non-reversible architectures, being applied in deep learning for memory savings and generativ…

cs.LG2024

WASH: Train your Ensemble with Communication-Efficient Weight Shuffling, then Average

Louis Fournier, Adel Nabli, Masih Aminbeidokhti +3

The performance of deep neural networks is enhanced by ensemble methods, which average the output of several models. However, this comes at an increased cost at inference. Weight a…