2 citations · 2 across the 6 of their papers we have counts for
4 papers · 1 filter
Learning Dynamics of Zeroth-Order Optimization: A Kernel Perspective
Zhe Li, Bicheng Ying, Zidong Liu +1
Classical optimization theory establishes that zeroth-order (ZO) algorithms suffer from a dimension-dependent slowdown, with convergence rates typically scaling with the model dime…
Can Language Models Compose Skills In-Context?
Zidong Liu, Zhuoyan Xu, Zhenmei Shi +1
Composing basic skills from simple tasks to accomplish composite tasks is crucial for modern intelligent systems. We investigate the in-context composition ability of language mode…
Converge Faster, Talk Less: Hessian-Informed Federated Zeroth-Order Optimization
Zhe Li, Bicheng Ying, Zidong Liu +2
Zeroth-order (ZO) optimization enables dimension-free communication in federated learning (FL), making it attractive for fine-tuning of large language models (LLMs) due to signific…
Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimization
Zhe Li, Bicheng Ying, Zidong Liu +2
Federated Learning (FL) offers a promising framework for collaborative and privacy-preserving machine learning across distributed data sources. However, the substantial communicati…