6 citations · 17 across the 8 of their papers we have counts for
8 papers
Generalized Polyak Step Size for First Order Optimization with Momentum
Xiaoyu Wang, Mikael Johansson, Tong Zhang
In machine learning applications, it is well known that carefully designed learning rate (step size) schedules can significantly improve the convergence of commonly used first-orde…
PAPAL: A Provable PArticle-based Primal-Dual ALgorithm for Mixed Nash Equilibrium
Shihong Ding, Hanze Dong, Cong Fang +2
We consider the non-convex non-concave objective function in two-player zero-sum continuous games. The existence of pure Nash equilibrium requires stringent conditions, posing a ma…
Particle-based Variational Inference with Preconditioned Functional Gradient Flow
Hanze Dong, Xi Wang, Yong Lin +1
Particle-based variational inference (VI) minimizes the KL divergence between model samples and the target posterior with gradient flow estimates. With the popularity of Stein vari…
Dimension Independent Generalization of DP-SGD for Overparameterized Smooth Convex Optimization
Yi-An Ma, Teodor Vanislavov Marinov, Tong Zhang
This paper considers the generalization performance of differentially private convex learning. We demonstrate that the convergence analysis of Langevin algorithms can be used to ob…
Beyond Uniform Lipschitz Condition in Differentially Private Optimization
Rudrajit Das, Satyen Kale, Zheng Xu +2
Most prior results on differentially private stochastic gradient descent (DP-SGD) are derived under the simplistic assumption of uniform Lipschitzness, i.e., the per-sample gradien…
Accelerated Dual-Averaging Primal-Dual Method for Composite Convex Minimization
Conghui Tan, Yuqiu Qian, Shiqian Ma +1
Dual averaging-type methods are widely used in industrial machine learning applications due to their ability to promoting solution structure (e.g., sparsity) efficiently. In this p…
Proximal Gradient Method for Nonsmooth Optimization over the Stiefel Manifold
Shixiang Chen, Shiqian Ma, Anthony Man-Cho So +1
We consider optimization problems over the Stiefel manifold whose objective function is the summation of a smooth function and a nonsmooth function. Existing methods for solving th…
On the Duality Gap Convergence of ADMM Methods
Da Tang, Tong Zhang
This paper provides a duality gap convergence analysis for the standard ADMM as well as a linearized version of ADMM. It is shown that under appropriate conditions, both methods ac…