1 citations · 1 across the 10 of their papers we have counts for
10 papers · 1 filter
Memory-Efficient Subspace Optimization with Residual Correction: From Momentum SGD to Muon
Sifan Yang, Hongyi Tao, Dingzhi Yu +5
Memory-efficient subspace optimization has emerged as a practical approach to reducing the memory cost of LLM training while preserving optimization performance. Previous algorithm…
Efficient Preconditioned Optimization Methods with Frequent Directions
Sifan Yang, Yuanyu Wan, Yibo Wang +1
Adaptive optimization methods have received considerable attention in recent years due to their strong empirical performance. However, they generally require maintaining a precondi…
Communication-Efficient Agnostic Federated Learning via Faster Convergence and Compression
Haomin Bai, Junyan Sun, Sifan Yang +2
Agnostic federated learning (AFL) seeks a model that performs reliably across heterogeneous workers, but communication remains a bottleneck. We improve communication efficiency…
From Switching to Dynamic Regret: A Simple Reduction via Unbiased Random Sequences
Yibo Wang, Wenhao Yang, Sifan Yang +3
In non-stationary online learning, dynamic regret has attracted increasing attention as a measure of how well an online learner performs against a time-varying comparator sequence.…
Efficient Multinomial Logistic Bandit via Frequent Directions
Linzhe He, Yu-Jie Zhang, Sifan Yang +1
This paper studies efficient online algorithms for multinomial logistic bandits (MLogB), where the feedback distribution over outcomes follows a multinomial logistic model of…
Distributed Online Convex Optimization with Compressed Communication: Optimal Regret and Applications
Sifan Yang, Dan-Yue Li, Lijun Zhang
Distributed online convex optimization (D-OCO) is a powerful paradigm for modeling distributed scenarios with streaming data. However, the communication cost between local learners…
Distributed Online Convex Optimization with Efficient Communication: Improved Algorithm and Lower bounds
Sifan Yang, Wenhao Yang, Wei Jiang +1
We investigate distributed online convex optimization with compressed communication, where learners connected by a network collaboratively minimize a sequence of global loss fu…
Online Nonsubmodular Optimization with Delayed Feedback in the Bandit Setting
Sifan Yang, Yuanyu Wan, Lijun Zhang
We investigate the online nonsubmodular optimization with delayed feedback in the bandit setting, where the loss function is -weakly DR-submodular and -weakly DR-supermodular…
Discounted Online Convex Optimization: Uniform Regret Across a Continuous Interval
Wenhao Yang, Sifan Yang, Lijun Zhang
Reflecting the greater significance of recent history over the distant past in non-stationary environments, -discounted regret has been introduced in online convex optimization…
Efficient Sign-Based Optimization: Accelerating Convergence via Variance Reduction
Wei Jiang, Sifan Yang, Wenhao Yang +1
Sign stochastic gradient descent (signSGD) is a communication-efficient method that transmits only the sign of stochastic gradients for parameter updating. Existing literature has…