2 papers
cs.LG2026
Demystifying Pipeline Parallelism: First Theory for PipeDream
Ivan Ilin, Peter Richtárik
Training modern machine learning models increasingly requires computation to be distributed across many accelerators. Data parallelism remains the default choice and is often paire…
cs.LG2026
Ringmaster LMO: Asynchronous Linear Minimization Oracle Momentum Method
Abdurakhmon Sadiev, Artavazd Maranjyan, Ivan Ilin +1
Muon has recently emerged as a strong alternative to AdamW for training neural networks, with encouraging large-scale pretraining results and growing evidence that matrix-structure…