Understanding and Exploiting Weight Update Sparsity for Communication-Efficient Distributed RL

Erfan Miahi, Eugene Belilovsky

Bandwidth-constrained distributed reinforcement learning (RL) post-training of large language models is bottlenecked by two channels: weight synchronization from trainers to infere…