4 citations · 12 across the 13 of their papers we have counts for
15 papers
Group Distributionally Robust Optimization over Decentralized Networks
Haomin Bai, Junyan Sun, Quan Cheng +2
We study group distributionally robust optimization (GDRO) over decentralized networks, where workers collaborate to learn robust models by minimizing the maximum local risk. Exist…
Open Problem: Is AdamW Effective Under Heavy-Tailed Noise?
Dingzhi Yu, Hongyi Tao, Yuanyu Wan +2
AdamW is the de facto optimizer for training large language models (LLMs), yet the theory behind it still lives mostly in finite-variance regimes. This is increasingly unsatisfying…
Distributed Online Convex Optimization with Compressed Communication: Optimal Regret and Applications
Sifan Yang, Dan-Yue Li, Lijun Zhang
Distributed online convex optimization (D-OCO) is a powerful paradigm for modeling distributed scenarios with streaming data. However, the communication cost between local learners…
Sign-Based Optimizers Are Effective Under Heavy-Tailed Noise
Dingzhi Yu, Hongyi Tao, Yuanyu Wan +2
While adaptive gradient methods are the workhorse of modern machine learning, sign-based optimization algorithms such as Lion and Muon have recently demonstrated superior empirical…
Online Nonsubmodular Optimization with Delayed Feedback in the Bandit Setting
Sifan Yang, Yuanyu Wan, Lijun Zhang
We investigate the online nonsubmodular optimization with delayed feedback in the bandit setting, where the loss function is -weakly DR-submodular and -weakly DR-supermodular…
Continuous Subspace Optimization for Continual Learning
Quan Cheng, Yuanyu Wan, Lingyu Wu +2
Continual learning aims to learn multiple tasks sequentially while preserving prior knowledge, but faces the challenge of catastrophic forgetting when adapting to new tasks. Recent…
Revisiting Projection-Free Online Learning with Time-Varying Constraints
Yibo Wang, Yuanyu Wan, Lijun Zhang
We investigate constrained online convex optimization, in which decisions must belong to a fixed and typically complicated domain, and are required to approximately satisfy additio…
Optimal and Efficient Algorithms for Decentralized Online Convex Optimization
Yuanyu Wan, Tong Wei, Bo Xue +2
We investigate decentralized online convex optimization (D-OCO), in which a set of local learners are required to minimize a sequence of global loss functions using only local comp…
Improved Regret for Bandit Convex Optimization with Delayed Feedback
Yuanyu Wan, Chang Yao, Mingli Song +1
We investigate bandit convex optimization (BCO) with delayed feedback, where only the loss value of the action is revealed under an arbitrary delay. Let denote the di…
Efficient Stochastic Approximation of Minimax Excess Risk Optimization
Lijun Zhang, Haomin Bai, Wei-Wei Tu +2
While traditional distributionally robust optimization (DRO) aims to minimize the maximal risk over a set of distributions, Agarwal and Zhang (2022) recently proposed a variant tha…
Non-stationary Projection-free Online Learning with Dynamic and Adaptive Regret Guarantees
Yibo Wang, Wenhao Yang, Wei Jiang +5
Projection-free online learning has drawn increasing interest due to its efficiency in solving high-dimensional problems with complicated constraints. However, most existing projec…
Non-stationary Delayed Online Convex Optimization: From Full-information to Bandit Setting
Yuanyu Wan, Chang Yao, Yitao Ma +2
Although online convex optimization (OCO) under arbitrary delays has received increasing attention recently, previous studies focus on stationary environments with the goal of mini…
Improved Dynamic Regret for Online Frank-Wolfe
Yuanyu Wan, Lijun Zhang, Mingli Song
To deal with non-stationary online problems with complex constraints, we investigate the dynamic regret of online Frank-Wolfe (OFW), which is an efficient projection-free algorithm…
Optimistic Online Mirror Descent for Bridging Stochastic and Adversarial Online Convex Optimization
Sijia Chen, Yu-Jie Zhang, Wei-Wei Tu +2
Stochastically Extended Adversarial (SEA) model is introduced by Sachs et al. [2022] as an interpolation between stochastic and adversarial online convex optimization. Under the sm…
Projection-free Online Learning with Arbitrary Delays
Yuanyu Wan, Yibo Wang, Chang Yao +2
Projection-free online learning, which eschews the projection operation via less expensive computations such as linear optimization (LO), has received much interest recently due to…