33 citations · 53 across the 14 of their papers we have counts for
15 papers
NBCLT: Parameter-Agnostic Norm-Ball Clipped Local Training
Linxuan Pan, Shucheng Yang, Junchi Yang
Distributed learning with multiple local steps reduces communication but aggravates drift, and its practical performance often depends on carefully tuned hyperparameters under hete…
Optimal High-Order Methods for Solving Monotone Variational Inequalities
Xinliang Zhang, Lesi Chen, Linxuan Pan +3
We study second- and higher-order methods for solving smooth monotone variational inequalities (MVI). Monteiro and Svaiter (SIAM J. Optim., 2012) showed that a second-order method,…
An Lower Bound for Stochastic NC-SC Bilevel Optimization with First-order Oracles
Zhihao Gu, Qilong Wu, Junchi Yang
We study the oracle complexity of finding -stationary points of smooth bilevel optimization problems with a nonconvex upper-level objective and a strongly convex lower-level pro…
SGHA: A Single-Loop Fully First-Order Algorithm for Nonconvex-Strongly-Convex Bilevel Optimization
Zhihao Gu, Qilong Wu, Junchi Yang
In this work, we study the oracle complexity of finding an -stationary point for nonconvex-strongly-convex (NC-SC) bilevel optimization using only first-order oracles. Existing…
Passive Acoustic Monitoring of Underwater Well Leakages with Machine Learning: A Review
Guanlin Zhu, Zechun Deng, Jiaxin Shen +1
Abandoned oil and gas wells pose significant environmental risks due to the potential leakage of hydrocarbons, brine and chemical pollutants. Detecting such leaks remains extremely…
A Pair of Diophantine Equations Involving the Fibonacci Numbers
Xuyuan Chen, Hung Viet Chu, Fadhlannafis K. Kesumajana +5
Let be relatively prime. Previous work showed that exactly one of the two equations and has a nonnegative…
Accelerating Distributed Optimization: A Primal-Dual Perspective on Local Steps
Junchi Yang, Murat Yildirim, Qiu Feng
In distributed machine learning, efficient training across multiple agents with different data distributions poses significant challenges. Even with a centralized coordinator, curr…
Parameter-Agnostic Optimization under Relaxed Smoothness
Florian Hübler, Junchi Yang, Xiang Li +1
Tuning hyperparameters, such as the stepsize, presents a major challenge of training machine learning models. To address this challenge, numerous adaptive optimization algorithms h…
Optimal Guarantees for Algorithmic Reproducibility and Gradient Complexity in Convex Optimization
Liang Zhang, Junchi Yang, Amin Karbasi +1
Algorithmic reproducibility measures the deviation in outputs of machine learning algorithms upon minor changes in the training process. Previous work suggests that first-order met…
Two Sides of One Coin: the Limits of Untuned SGD and the Power of Adaptive Methods
Junchi Yang, Xiang Li, Ilyas Fatkhullin +1
The classical analysis of Stochastic Gradient Descent (SGD) with polynomially decaying stepsize relies on well-tuned depending on problem parameters such as…
TiAda: A Time-scale Adaptive Algorithm for Nonconvex Minimax Optimization
Xiang Li, Junchi Yang, Niao He
Adaptive gradient methods have shown their ability to adjust the stepsizes on the fly in a parameter-agnostic manner, and empirically achieve faster convergence for solving minimiz…
Nest Your Adaptive Algorithm for Parameter-Agnostic Nonconvex Minimax Optimization
Junchi Yang, Xiang Li, Niao He
Adaptive algorithms like AdaGrad and AMSGrad are successful in nonconvex optimization owing to their parameter-agnostic ability -- requiring no a priori knowledge about problem-spe…
Faster Single-loop Algorithms for Minimax Optimization without Strong Concavity
Junchi Yang, Antonio Orvieto, Aurelien Lucchi +1
Gradient descent ascent (GDA), the simplest single-loop algorithm for nonconvex minimax optimization, is widely used in practical applications such as generative adversarial networ…
The Complexity of Nonconvex-Strongly-Concave Minimax Optimization
Siqi Zhang, Junchi Yang, Cristóbal Guzmán +2
This paper studies the complexity for finding approximate stationary points of nonconvex-strongly-concave (NC-SC) smooth minimax problems, in both general and averaged smooth finit…
Global Convergence and Variance-Reduced Optimization for a Class of Nonconvex-Nonconcave Minimax Problems
Junchi Yang, Negar Kiyavash, Niao He
Nonconvex minimax problems appear frequently in emerging machine learning applications, such as generative adversarial networks and adversarial learning. Simple algorithms such as…