4 papers
Compander-Aligned Query Geometry for Quantized Zeroth-Order Optimization
Yao Shu, Zilin Zhu
Low-bit forward evaluation is an attractive route to memory-efficient zeroth-order (ZO) adaptation: the optimizer needs only scalar losses, and the model can be queried near deploy…
Why Zeroth-Order Adaptation May Forget Less: A Randomized Shaping Theory
Yao Shu, Jian Mu, Zhongxiang Dai
Continual learning requires new-task adaptation without damaging previously acquired capabilities. Recent forward-pass and zeroth-order (ZO) results show that low-query adaptation…
Meta-Prompt Optimization for LLM-Based Sequential Decision Making
Mingze Kong, Zhiyong Wang, Yao Shu +1
Large language models (LLMs) have recently been employed as agents to solve sequential decision-making tasks such as Bayesian optimization and multi-armed bandits (MAB). These work…
Prompt Optimization with EASE? Efficient Ordering-aware Automated Selection of Exemplars
Zhaoxuan Wu, Xiaoqiang Lin, Zhongxiang Dai +5
Large language models (LLMs) have shown impressive capabilities in real-world applications. The capability of in-context learning (ICL) allows us to adapt an LLM to downstream task…