ElasticZO: A Memory-Efficient On-Device Learning with Combined Zeroth- and First-Order Optimization

Keisuke Sugiura, Hiroki Matsutani

Zeroth-order (ZO) optimization is being recognized as a simple yet powerful alternative to standard backpropagation (BP)-based training. Notably, ZO optimization allows for trainin…