11 citations · 37 across the 19 of their papers we have counts for
19 papers
SPARKLE: A Unified Single-Loop Primal-Dual Framework for Decentralized Bilevel Optimization
Shuchen Zhu, Boao Kong, Songtao Lu +2
This paper studies decentralized bilevel optimization, in which multiple agents collaborate to solve problems involving nested optimization structures with neighborhood communicati…
A Mathematics-Inspired Learning-to-Optimize Framework for Decentralized Optimization
Yutong He, Qiulin Shang, Xinmeng Huang +2
Most decentralized optimization algorithms are handcrafted. While endowed with strong theoretical guarantees, these algorithms generally target a broad class of problems, thereby n…
Distributed Bilevel Optimization with Communication Compression
Yutong He, Jie Hu, Xinmeng Huang +3
Stochastic bilevel optimization tackles challenges involving nested optimization structures. Its fast-growing scale nowadays necessitates efficient distributed algorithms. In conve…
Decentralized Bilevel Optimization: A Perspective from Transient Iteration Complexity
Boao Kong, Shuchen Zhu, Songtao Lu +2
Stochastic bilevel optimization (SBO) is becoming increasingly essential in machine learning due to its versatility in handling nested structures. To address large-scale SBO, decen…
Understanding the Influence of Digraphs on Decentralized Optimization: Effective Metrics, Lower Bound, and Optimal Algorithm
Liyuan Liang, Xinmeng Huang, Ran Xin +1
This paper investigates the influence of directed networks on decentralized stochastic non-convex optimization associated with column-stochastic mixing matrices. Surprisingly, we f…
Stochastic Controlled Averaging for Federated Learning with Communication Compression
Xinmeng Huang, Ping Li, Xiaoyun Li
Communication compression, a technique aiming to reduce the information volume to be transmitted over the air, has gained great interests in Federated Learning (FL) for the potenti…
Momentum Benefits Non-IID Federated Learning Simply and Provably
Ziheng Cheng, Xinmeng Huang, Pengfei Wu +1
Federated learning is a powerful paradigm for large-scale machine learning, but it faces significant challenges due to unreliable network connections, slow communication, and subst…
Optimal Multitask Linear Regression and Contextual Bandits under Sparse Heterogeneity
Xinmeng Huang, Kan Xu, Donghwan Lee +3
Large and complex datasets are often collected from several, possibly heterogeneous sources. Multitask learning methods improve efficiency by leveraging commonalities across datase…
Unbiased Compression Saves Communication in Distributed Optimization: When and How Much?
Yutong He, Xinmeng Huang, Kun Yuan
Communication compression is a common technique in distributed optimization that can alleviate communication overhead by transmitting compressed gradients and model parameters. How…
Demystifying Disagreement-on-the-Line in High Dimensions
Donghwan Lee, Behrad Moniri, Xinmeng Huang +2
Evaluating the performance of machine learning models under distribution shift is challenging, especially when we only have unlabeled data from the shifted (target) domain, along w…
Revisiting Optimal Convergence Rate for Smooth and Non-convex Stochastic Decentralized Optimization
Kun Yuan, Xinmeng Huang, Yiming Chen +3
Decentralized optimization is effective to save communication in large-scale machine learning. Although numerous algorithms have been proposed with theoretical guarantees and empir…
Lower Bounds and Nearly Optimal Algorithms in Distributed Learning with Communication Compression
Xinmeng Huang, Yiming Chen, Wotao Yin +1
Recent advances in distributed optimization and learning have shown that communication compression is one of the most effective means of reducing communication. While there have be…
Collaborative Learning of Discrete Distributions under Heterogeneity and Communication Constraints
Xinmeng Huang, Donghwan Lee, Edgar Dobriban +1
In modern machine learning, users often have to collaborate to learn the distribution of the data. Communication can be a significant bottleneck. Prior work has studied homogeneous…
T-Cal: An optimal test for the calibration of predictive models
Donghwan Lee, Xinmeng Huang, Hamed Hassani +1
The prediction accuracy of machine learning methods is steadily increasing, but the calibration of their uncertainty predictions poses a significant challenge. Numerous works focus…
Removing Data Heterogeneity Influence Enhances Network Topology Dependence of Decentralized SGD
Kun Yuan, Sulaiman A. Alghunaim, Xinmeng Huang
We consider the decentralized stochastic optimization problems, where a network of nodes, each owning a local cost function, cooperate to find a minimizer of the globally-avera…
DecentLaM: Decentralized Momentum SGD for Large-batch Deep Training
Kun Yuan, Yiming Chen, Xinmeng Huang +4
The scale of deep learning nowadays calls for efficient distributed training algorithms. Decentralized momentum SGD (DmSGD), in which each node averages only with its neighbors, is…
Improved Analysis and Rates for Variance Reduction under Without-replacement Sampling Orders
Xinmeng Huang, Kun Yuan, Xianghui Mao +1
When applying a stochastic algorithm, one must choose an order to draw samples. The practical choices are without-replacement sampling orders, which are empirically faster and more…
Scaled Relative Graph of Normal Matrices
Xinmeng Huang, Ernest K. Ryu, Wotao Yin
The Scaled Relative Graph (SRG) is a geometric tool that maps the action of a multi-valued nonlinear operator onto the 2D plane, used to analyze the convergence of a wide range of…
Tight Coefficients of Averaged Operators via Scaled Relative Graph
Xinmeng Huang, Ernest K. Ryu, Wotao Yin
Many iterative methods in optimization are fixed-point iterations with averaged operators. As such methods converge at an rate with the constant determined by th…