Elucidating Subspace Perturbation in Zeroth-Order Optimization: Theory and Practice at Scale

Sihwan Park, Jihun Yun, SungYub Kim +2

Zeroth-order (ZO) optimization has emerged as a promising alternative to gradient-based backpropagation methods, particularly for black-box optimization and large language model (L…