66 citations · 159 across the 21 of their papers we have counts for
21 papers
Convergence Analysis of Muon-type Methods with Inexact LMO in the Degenerate Case
Xun Qian, Peter Richtárik
Muon-type methods have demonstrated potentially superior performance over Adam and its variants, and have shown hyperparameter transferability across model sizes when specific norm…
Broximal Alignment for Global Non-Convex Optimization
Kaja Gruntkowska, Hanmin Li, Xun Qian +1
Most non-convex optimization theory is built around gradient dynamics, leaving global convergence largely unexplored. The dominant paradigm focuses on stationarity, certifying only…
Communication-Efficient Gluon in Federated Learning
Xun Qian, Alexander Gaponov, Grigory Malinovsky +1
Recent developments have shown that Muon-type optimizers based on linear minimization oracles (LMOs) over non-Euclidean norm balls have the potential to get superior practical perf…
Muon is Provably Faster with Momentum Variance Reduction
Xun Qian, Hussein Rammal, Dmitry Kovalev +1
Recent empirical research has demonstrated that deep learning optimizers based on the linear minimization oracle (LMO) over specifically chosen Non-Euclidean norm balls, such as Mu…
A matrix-free interior point continuous trajectory for linearly constrained convex programming
Xun Qian, Li-Zhi Liao, Jie Sun
Interior point methods for solving linearly constrained convex programming involve a variable projection matrix at each iteration to deal with the linear constraints. This matrix o…
A Stochastic Block-coordinate Proximal Newton Method for Nonconvex Composite Minimization
Hong Zhu, Xun Qian
This paper presents a stochastic block-coordinate proximal Newton method for minimizing the sum of a blockwise Lipschitz-continuously differentiable function and a separable nonsmo…
Deep Learning for Multi-Country GDP Prediction: A Study of Model Performance and Data Impact
Huaqing Xie, Xingcheng Xu, Fangjia Yan +2
GDP is a vital measure of a country's economic health, reflecting the total value of goods and services produced. Forecasting GDP growth is essential for economic planning, as it h…
DocReLM: Mastering Document Retrieval with Language Model
Gengchen Wei, Xinle Pang, Tianning Zhang +5
With over 200 million published academic documents and millions of new documents being written each year, academic researchers face the challenge of searching for information withi…
Communication-Efficient Distributed Learning with Local Immediate Error Compensation
Yifei Cheng, Li Shen, Linli Xu +6
Gradient compression with error compensation has attracted significant attention with the target of reducing the heavy communication overhead in distributed learning. However, exis…
Catalyst Acceleration of Error Compensated Methods Leads to Better Communication Complexity
Xun Qian, Hanze Dong, Tong Zhang +1
Communication overhead is well known to be a key bottleneck in large scale distributed learning, and a particularly successful class of methods which help to overcome this bottlene…
Distributed Newton-Type Methods with Communication Compression and Bernoulli Aggregation
Rustem Islamov, Xun Qian, Slavomír Hanzely +2
Despite their high computation and communication costs, Newton-type methods remain an appealing option for distributed training due to their robustness against ill-conditioned conv…
Basis Matters: Better Communication-Efficient Second Order Methods for Federated Learning
Xun Qian, Rustem Islamov, Mher Safaryan +1
Recent advances in distributed optimization have shown that Newton-type methods with proper communication compression mechanisms can guarantee fast local rates and low communicatio…
Error Compensated Loopless SVRG, Quartz, and SDCA for Distributed Optimization
Xun Qian, Hanze Dong, Peter Richtárik +1
The communication of gradients is a key bottleneck in distributed training of large scale machine learning models. In order to reduce the communication cost, gradient compression (…
FedNL: Making Newton-Type Methods Applicable to Federated Learning
Mher Safaryan, Rustem Islamov, Xun Qian +1
Inspired by recent work of Islamov et al (2021), we propose a family of Federated Newton Learn (FedNL) methods, which we believe is a marked step in the direction of making second-…
Distributed Second Order Methods with Fast Rates and Compressed Communication
Rustem Islamov, Xun Qian, Peter Richtárik
We develop several new communication-efficient second-order methods for distributed optimization. Our first method, NEWTON-STAR, is a variant of Newton's method from which it inher…
Error Compensated Distributed SGD Can Be Accelerated
Xun Qian, Peter Richtárik, Tong Zhang
Gradient compression is a recent and increasingly popular technique for reducing the communication cost in distributed training of large-scale machine learning models. In this work…
Acceleration for Compressed Gradient Descent in Distributed and Federated Optimization
Zhize Li, Dmitry Kovalev, Xun Qian +1
Due to the high communication cost in distributed and federated learning problems, methods relying on compression of communicated messages are becoming increasingly popular. While…
L-SVRG and L-Katyusha with Arbitrary Sampling
Xun Qian, Zheng Qu, Peter Richtárik
We develop and analyze a new family of {\em nonaccelerated and accelerated loopless variance-reduced methods} for finite sum optimization problems. Our convergence analysis relies…
MISO is Making a Comeback With Better Proofs and Rates
Xun Qian, Alibek Sailanbayev, Konstantin Mishchenko +1
MISO, also known as Finito, was one of the first stochastic variance reduced methods discovered, yet its popularity is fairly low. Its initial analysis was significantly limited by…
SAGA with Arbitrary Sampling
Xu Qian, Zheng Qu, Peter Richtárik
We study the problem of minimizing the average of a very large number of smooth functions, which is of key importance in training supervised learning models. One of the most celebr…
SGD: General Analysis and Improved Rates
Robert Mansel Gower, Nicolas Loizou, Xun Qian +3
We propose a general yet simple theorem describing the convergence of SGD under the arbitrary sampling paradigm. Our theorem describes the convergence of an infinite array of varia…