5 papers · 1 filter
Group Distributionally Robust Optimization over Decentralized Networks
Haomin Bai, Junyan Sun, Quan Cheng +2
We study group distributionally robust optimization (GDRO) over decentralized networks, where workers collaborate to learn robust models by minimizing the maximum local risk. Exist…
Memory-Efficient Subspace Optimization with Residual Correction: From Momentum SGD to Muon
Sifan Yang, Hongyi Tao, Dingzhi Yu +5
Memory-efficient subspace optimization has emerged as a practical approach to reducing the memory cost of LLM training while preserving optimization performance. Previous algorithm…
Last-Iterate Convergence of Clipped SGD with Linear Decay
Wei Jiang, Haomin Bai, Zechao Li
In this paper, we study the last-iterate convergence of clipped stochastic gradient descent under heavy-tailed noise. We assume that the stochastic gradients are unbiased and that…
Dimension-Free Decentralized Nonsmooth Nonconvex Stochastic Optimization
Yuanyu Wan, Lan Xue, Haomin Bai +2
We investigate decentralized nonsmooth nonconvex stochastic optimization over a network of nodes, with the goal of finding an -Goldstein stationary point. The best exist…
Communication-Efficient Agnostic Federated Learning via Faster Convergence and Compression
Haomin Bai, Junyan Sun, Sifan Yang +2
Agnostic federated learning (AFL) seeks a model that performs reliably across heterogeneous workers, but communication remains a bottleneck. We improve communication efficiency…