3 papers
cs.LG2025
MT-DAO: Multi-Timescale Distributed Adaptive Optimizers with Local Updates
Alex Iacob, Andrej Jovanovic, Mher Safaryan +6
Training large models with distributed data parallelism (DDP) requires frequent communication of gradients across workers, which can saturate bandwidth. Infrequent communication st…
cs.LG2025
DES-LOC: Desynced Low Communication Adaptive Optimizers for Training Foundation Models
Alex Iacob, Lorenzo Sani, Mher Safaryan +8
Scaling foundation model training with Distributed Data Parallel (DDP) methods is bandwidth-limited. Existing infrequent communication methods like Local SGD were designed to synch…
cs.LG2025
Editing as Unlearning: Are Knowledge Editing Methods Strong Baselines for Large Language Model Unlearning?
Zexi Li, Xiangzhu Wang, William F. Shen +5
Large language Model (LLM) unlearning, i.e., selectively removing information from LLMs, is vital for responsible model deployment. Differently, LLM knowledge editing aims to modif…