Why Zeroth-Order Adaptation May Forget Less: A Randomized Shaping Theory

Yao Shu, Jian Mu, Zhongxiang Dai

Continual learning requires new-task adaptation without damaging previously acquired capabilities. Recent forward-pass and zeroth-order (ZO) results show that low-query adaptation…