1 citations · 1 across the 10 of their papers we have counts for
13 papers
Memory-Efficient Subspace Optimization with Residual Correction: From Momentum SGD to Muon
Sifan Yang, Hongyi Tao, Dingzhi Yu +5
Memory-efficient subspace optimization has emerged as a practical approach to reducing the memory cost of LLM training while preserving optimization performance. Previous algorithm…
Efficient Preconditioned Optimization Methods with Frequent Directions
Sifan Yang, Yuanyu Wan, Yibo Wang +1
Adaptive optimization methods have received considerable attention in recent years due to their strong empirical performance. However, they generally require maintaining a precondi…
Communication-Efficient Agnostic Federated Learning via Faster Convergence and Compression
Haomin Bai, Junyan Sun, Sifan Yang +2
Agnostic federated learning (AFL) seeks a model that performs reliably across heterogeneous workers, but communication remains a bottleneck. We improve communication efficiency…
From Switching to Dynamic Regret: A Simple Reduction via Unbiased Random Sequences
Yibo Wang, Wenhao Yang, Sifan Yang +3
In non-stationary online learning, dynamic regret has attracted increasing attention as a measure of how well an online learner performs against a time-varying comparator sequence.…
Efficient Multinomial Logistic Bandit via Frequent Directions
Linzhe He, Yu-Jie Zhang, Sifan Yang +1
This paper studies efficient online algorithms for multinomial logistic bandits (MLogB), where the feedback distribution over outcomes follows a multinomial logistic model of…
Distributed Online Convex Optimization with Compressed Communication: Optimal Regret and Applications
Sifan Yang, Dan-Yue Li, Lijun Zhang
Distributed online convex optimization (D-OCO) is a powerful paradigm for modeling distributed scenarios with streaming data. However, the communication cost between local learners…
Distributed Online Convex Optimization with Efficient Communication: Improved Algorithm and Lower bounds
Sifan Yang, Wenhao Yang, Wei Jiang +1
We investigate distributed online convex optimization with compressed communication, where learners connected by a network collaboratively minimize a sequence of global loss fu…
Online Nonsubmodular Optimization with Delayed Feedback in the Bandit Setting
Sifan Yang, Yuanyu Wan, Lijun Zhang
We investigate the online nonsubmodular optimization with delayed feedback in the bandit setting, where the loss function is -weakly DR-submodular and -weakly DR-supermodular…
Better Convergence Guarantees for Sign-Based Momentum Methods
Wei Jiang, Dingzhi Yu, Sifan Yang +3
This paper presents an improved analysis for sign-based methods with momentum updates. Traditional sign-based methods obtain a convergence rate of under the…
Discounted Online Convex Optimization: Uniform Regret Across a Continuous Interval
Wenhao Yang, Sifan Yang, Lijun Zhang
Reflecting the greater significance of recent history over the distant past in non-stationary environments, -discounted regret has been introduced in online convex optimization…
Projection-Free Variance Reduction Methods for Stochastic Constrained Multi-Level Compositional Optimization
Wei Jiang, Sifan Yang, Wenhao Yang +3
This paper investigates projection-free algorithms for stochastic constrained multi-level optimization. In this context, the objective function is a nested composition of several s…
Adaptive Variance Reduction for Stochastic Optimization under Weaker Assumptions
Wei Jiang, Sifan Yang, Yibo Wang +1
This paper explores adaptive variance reduction methods for stochastic optimization based on the STORM technique. Existing adaptive extensions of STORM rely on strong assumptions l…
Efficient Sign-Based Optimization: Accelerating Convergence via Variance Reduction
Wei Jiang, Sifan Yang, Wenhao Yang +1
Sign stochastic gradient descent (signSGD) is a communication-efficient method that transmits only the sign of stochastic gradients for parameter updating. Existing literature has…