10 citations · 32 across the 20 of their papers we have counts for
23 papers
Revisiting Smoothed Online Learning: Competitive Ratio, Dynamic Regret with Switching Cost, and Best of Both Worlds
Lijun Zhang, Linzhe He, Wei Jiang +1
We revisit smoothed online learning and develop algorithms with improved competitive ratios, optimal dynamic regret with switching cost, and best-of-both-worlds guarantees. Surpris…
Memory-Efficient Subspace Optimization with Residual Correction: From Momentum SGD to Muon
Sifan Yang, Hongyi Tao, Dingzhi Yu +5
Memory-efficient subspace optimization has emerged as a practical approach to reducing the memory cost of LLM training while preserving optimization performance. Previous algorithm…
Last-Iterate Convergence of Clipped SGD with Linear Decay
Wei Jiang, Haomin Bai, Zechao Li
In this paper, we study the last-iterate convergence of clipped stochastic gradient descent under heavy-tailed noise. We assume that the stochastic gradients are unbiased and that…
Approximate Risk Minimization Over Shrinking-Thresholding Rules in Normal Mean Estimation
Wei Jiang
We develop an approximate risk minimization framework for shrinkage-thresholding estimation in normal mean problems. In the canonical multivariate normal mean model, we introduce a…
Distributed Online Convex Optimization with Efficient Communication: Improved Algorithm and Lower bounds
Sifan Yang, Wenhao Yang, Wei Jiang +1
We investigate distributed online convex optimization with compressed communication, where learners connected by a network collaboratively minimize a sequence of global loss fu…
Convergence Analysis of the Lion Optimizer in Centralized and Distributed Settings
Wei Jiang, Lijun Zhang
In this paper, we analyze the convergence properties of the Lion optimizer. First, we establish that the Lion optimizer attains a convergence rate of …
Dual Adaptivity: Universal Algorithms for Minimizing the Adaptive Regret of Convex Functions
Lijun Zhang, Wenhao Yang, Guanghui Wang +2
To deal with changing environments, a new performance measure -- adaptive regret, defined as the maximum static regret over any interval, was proposed in online learning. Under the…
Better Convergence Guarantees for Sign-Based Momentum Methods
Wei Jiang, Dingzhi Yu, Sifan Yang +3
This paper presents an improved analysis for sign-based methods with momentum updates. Traditional sign-based methods obtain a convergence rate of under the…
Mirror Descent Under Generalized Smoothness
Dingzhi Yu, Wei Jiang, Hongyi Tao +2
Smoothness is crucial for attaining fast rates in first-order optimization. However, many optimization problems in modern machine learning involve non-smooth objectives. Recent stu…
Projection-Free Variance Reduction Methods for Stochastic Constrained Multi-Level Compositional Optimization
Wei Jiang, Sifan Yang, Wenhao Yang +3
This paper investigates projection-free algorithms for stochastic constrained multi-level optimization. In this context, the objective function is a nested composition of several s…
Adaptive Variance Reduction for Stochastic Optimization under Weaker Assumptions
Wei Jiang, Sifan Yang, Yibo Wang +1
This paper explores adaptive variance reduction methods for stochastic optimization based on the STORM technique. Existing adaptive extensions of STORM rely on strong assumptions l…
Efficient Sign-Based Optimization: Accelerating Convergence via Variance Reduction
Wei Jiang, Sifan Yang, Wenhao Yang +1
Sign stochastic gradient descent (signSGD) is a communication-efficient method that transmits only the sign of stochastic gradients for parameter updating. Existing literature has…
Efficient Algorithms for Empirical Group Distributionally Robust Optimization and Beyond
Dingzhi Yu, Yunuo Cai, Wei Jiang +1
In this paper, we investigate the empirical counterpart of Group Distributionally Robust Optimization (GDRO), which aims to minimize the maximal empirical risk across distinct…
Learning Unnormalized Statistical Models via Compositional Optimization
Wei Jiang, Jiayu Qin, Lingyu Wu +3
Learning unnormalized statistical models (e.g., energy-based models) is computationally challenging due to the complexity of handling the partition function. To eschew this complex…
Evolving Testing Scenario Generation Method and Intelligence Evaluation Framework for Automated Vehicles
Yining Ma, Wei Jiang, Lingtong Zhang +5
Interaction between the background vehicles (BVs) and automated vehicles (AVs) in scenario-based testing plays a critical role in evaluating the intelligence of the AVs. Current te…
Non-stationary Projection-free Online Learning with Dynamic and Adaptive Regret Guarantees
Yibo Wang, Wenhao Yang, Wei Jiang +5
Projection-free online learning has drawn increasing interest due to its efficiency in solving high-dimensional problems with complicated constraints. However, most existing projec…
A Privacy-Preserving Finite-Time Push-Sum based Gradient Method for Distributed Optimization over Digraphs
Xiaomeng Chen, Wei Jiang, Themistoklis Charalambous +1
This paper addresses the problem of distributed optimization, where a network of agents represented as a directed graph (digraph) aims to collaboratively minimize the sum of their…
Multi-block-Single-probe Variance Reduced Estimator for Coupled Compositional Optimization
Wei Jiang, Gang Li, Yibo Wang +2
Variance reduction techniques such as SPIDER/SARAH/STORM have been extensively studied to improve the convergence rates of stochastic non-convex optimization, which usually maintai…
Smoothed Online Convex Optimization Based on Discounted-Normal-Predictor
Lijun Zhang, Wei Jiang, Jinfeng Yi +1
In this paper, we investigate an online prediction strategy named as Discounted-Normal-Predictor (Kapralov and Panigrahy, 2010) for smoothed online convex optimization (SOCO), in w…
Optimal Algorithms for Stochastic Multi-Level Compositional Optimization
Wei Jiang, Bokun Wang, Yibo Wang +2
In this paper, we investigate the problem of stochastic multi-level compositional optimization, where the objective function is a composition of multiple smooth but possibly non-co…
Revisiting Smoothed Online Learning
Lijun Zhang, Wei Jiang, Shiyin Lu +1
In this paper, we revisit the problem of smoothed online learning, in which the online learner suffers both a hitting cost and a switching cost, and target two performance metrics:…
ACNe: Attentive Context Normalization for Robust Permutation-Equivariant Learning
Weiwei Sun, Wei Jiang, Eduard Trulls +2
Many problems in computer vision require dealing with sparse, unordered data in the form of point clouds. Permutation-equivariant networks have become a popular solution-they opera…
Dual Adaptivity: A Universal Algorithm for Minimizing the Adaptive Regret of Convex Functions
Lijun Zhang, Guanghui Wang, Wei-Wei Tu +1
To deal with changing environments, a new performance measure -- adaptive regret, defined as the maximum static regret over any interval, was proposed in online learning. Under the…