3 papers
cs.LG2026
Dominant-Layer ZO: A Single Layer Dominates Zeroth-Order Fine-Tuning of LLMs
Wanhao Yu, Ziyan Wang, Zheng Wang +7
Zeroth-order (ZO) optimization enables memory-efficient fine-tuning of large language models (LLMs) using only forward passes, but it remains unclear how useful adaptation is distr…
cs.LG2026
More Than Memory Savings: Zeroth-Order Optimization Mitigates Forgetting in Continual Learning
Wanhao Yu, Zheng Wang, Shuteng Niu +2
Zeroth-order (ZO) optimization has gained attention as a memory-efficient alternative to first-order (FO) methods, particularly in settings where gradient computation is expensive…
cs.LG2026
Rethinking Continual Learning with Progressive Neural Collapse
Zheng Wang, Wanhao Yu, Li Yang +1
Continual Learning (CL) seeks to build an agent that can continuously learn a sequence of tasks, where a key challenge, namely Catastrophic Forgetting, persists due to the potentia…