3.7k citations · 8k across the 129 of their papers we have counts for
133 papers
StructZO: Structured Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning
Xuanlin Yang, Jia Li, Zhouchen Lin
Fine-tuning large language models at scale is increasingly limited by the memory cost of backpropagation. Zeroth-Order (ZO) optimizers reduce this burden by relying only on forward…
SpecGradFilter: A Spectral Gradient Filtering Framework for Taming Federated Heterogeneity
Liyang Yuan, Yibo Yang, Dandan Guo +2
Federated Learning (FL) is fundamentally challenged by statistical heterogeneity, where non-identically distributed (non-IID) data induces client drift that severely hampers global…
Near-Optimal Lower Bounds for Randomized Algorithms in Exact Value Zeroth-Order Convex Optimization
Haihan Zhang, Chenheng Zhang, Zhiquan Qi +1
Whether exact scalar feedback intrinsically incurs the additional dimension paid by known zeroth-order methods remains open even for Lipschitz convex optimization. For a univer…
Neural Collapse Terminus: A Unified Solution for Class Incremental Learning and Its Variants
Yibo Yang, Haobo Yuan, Xiangtai Li +6
How to enable learnability for new classes while keeping the capability well on old classes has been a crucial challenge for class incremental learning. Beyond the normal case, lon…
Zeroth-order Optimization with Weak Dimension Dependency
Pengyun Yue, Long Yang, Cong Fang +1
Zeroth-order optimization is a fundamental research topic that has been a focus of various learning tasks, such as black-box adversarial attacks, bandits, and reinforcement learnin…
ShadowNet for Data-Centric Quantum System Learning
Yuxuan Du, Yibo Yang, Tongliang Liu +3
Understanding the dynamics of large quantum systems is hindered by the curse of dimensionality. Statistical learning offers new possibilities in this regime through neural network…