Private Zeroth-Order Nonsmooth Nonconvex Optimization

Abstract

We introduce a new zeroth-order algorithm for private stochastic optimization on nonconvex and nonsmooth objectives. Given a dataset of size $M$, our algorithm ensures $(\hat{I}\pm ,\hat{I}\pm \ddot{I}{}^2/2)$-Rényi differential privacy and finds a $(\hat{I}´,\hat{I}\mu )$-stationary point so long as $M=\tilde{\hat{I}}\text{c◯}\left(\frac{d}{\hat{I}´\hat{I}{\mu }^3}+\frac{d^{3/2}}{\ddot{I}\hat{I}´\hat{I}{\mu }^2}\right)$. This matches the optimal complexity of its non-private zeroth-order analog. Notably, although the objective is not smooth, we have privacy ``for free'' whenever $\ddot{I}\ge \sqrt{d}\hat{I}\mu$.