Perturbation-efficient Zeroth-order Optimization for Hardware-friendly On-device Training

Qitao Tan, Sung-En Chang, Rui Xia +10

Zeroth-order (ZO) optimization is an emerging deep neural network (DNN) training paradigm that offers computational simplicity and memory savings. However, this seemingly promising…