Zeroth-Order Optimization at the Edge of Stability

Minhak Song, Liang Zhang, Bingcong Li +3

Zeroth-order (ZO) methods are widely used when gradients are unavailable or prohibitively expensive, including black-box learning and memory-efficient fine-tuning of large models,…