VAMO: Efficient Zeroth-Order Variance Reduction for SGD with Faster Convergence

Jiahe Chen, Ziye Ma

Optimizing large-scale nonconvex problems, common in deep learning, demands balancing rapid convergence with computational efficiency. First-order (FO) optimizers, which serve as t…