5 papers · 1 filter
Distributed Online Convex Optimization with Efficient Communication: Improved Algorithm and Lower bounds
Sifan Yang, Wenhao Yang, Wei Jiang +1
We investigate distributed online convex optimization with compressed communication, where learners connected by a network collaboratively minimize a sequence of global loss fu…
Convergence Analysis of the Lion Optimizer in Centralized and Distributed Settings
Wei Jiang, Lijun Zhang
In this paper, we analyze the convergence properties of the Lion optimizer. First, we establish that the Lion optimizer attains a convergence rate of …
Dual Adaptivity: Universal Algorithms for Minimizing the Adaptive Regret of Convex Functions
Lijun Zhang, Wenhao Yang, Guanghui Wang +2
To deal with changing environments, a new performance measure -- adaptive regret, defined as the maximum static regret over any interval, was proposed in online learning. Under the…
Efficient Sign-Based Optimization: Accelerating Convergence via Variance Reduction
Wei Jiang, Sifan Yang, Wenhao Yang +1
Sign stochastic gradient descent (signSGD) is a communication-efficient method that transmits only the sign of stochastic gradients for parameter updating. Existing literature has…
Efficient Algorithms for Empirical Group Distributionally Robust Optimization and Beyond
Dingzhi Yu, Yunuo Cai, Wei Jiang +1
In this paper, we investigate the empirical counterpart of Group Distributionally Robust Optimization (GDRO), which aims to minimize the maximal empirical risk across distinct…