1 citations · 1 across the 11 of their papers we have counts for
5 papers · 1 filter
SUDA-Muon: Structural Design Principles and Boundaries for Fully Decentralized Muon
Hengrui Zhang, Boao Kong, Jiahe Geng +1
Fully decentralized Muon is difficult because its nonlinear matrix-sign operator does not commute with linear gossip averaging. This makes decentralized Muon a structural design pr…
On the Convergence of Stochastic Gradient Descent with Perturbed Forward-Backward Passes
Boao Kong, Hengrui Zhang, Kun Yuan
We study stochastic gradient descent (SGD) for composite optimization problems with sequential operators subject to perturbations in both the forward and backward passes. Unlik…
Clapping: Removing Per-sample Storage for Pipeline Parallel Distributed Optimization with Communication Compression
Boao Kong, Xu Huang, Yuqi Xu +3
Pipeline-parallel distributed optimization is essential for large-scale machine learning but is challenged by significant communication overhead from transmitting high-dimensional…
SPARKLE: A Unified Single-Loop Primal-Dual Framework for Decentralized Bilevel Optimization
Shuchen Zhu, Boao Kong, Songtao Lu +2
This paper studies decentralized bilevel optimization, in which multiple agents collaborate to solve problems involving nested optimization structures with neighborhood communicati…
Decentralized Bilevel Optimization: A Perspective from Transient Iteration Complexity
Boao Kong, Shuchen Zhu, Songtao Lu +2
Stochastic bilevel optimization (SBO) is becoming increasingly essential in machine learning due to its versatility in handling nested structures. To address large-scale SBO, decen…