Why Does Adaptive Zeroth-Order Optimization Work?

Haishan Ye, Luo Luo

Zeroth-order (ZO) optimization is popular in real-world applications that accessing the gradient information is expensive or unavailable. Recently, adaptive ZO methods that normali…