34 citations · 70 across the 5 of their papers we have counts for
7 papers · 1 filter
Toward Communication Efficient Adaptive Gradient Method
Xiangyi Chen, Xiaoyun Li, Ping Li
In recent years, distributed optimization is proven to be an effective approach to accelerate training of large scale machine learning models such as deep neural networks. With the…
On the Convergence of Decentralized Adaptive Gradient Methods
Xiangyi Chen, Belhal Karimi, Weijie Zhao +1
Adaptive gradient methods including Adam, AdaGrad, and their variants have been very successful for training deep learning models, such as neural networks. Meanwhile, given the nee…
Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacy
Xinwei Zhang, Xiangyi Chen, Mingyi Hong +2
Providing privacy protection has been one of the primary motivations of Federated Learning (FL). Recently, there has been a line of work on incorporating the formal privacy notion…
Understanding Gradient Clipping in Private SGD: A Geometric Perspective
Xiangyi Chen, Zhiwei Steven Wu, Mingyi Hong
Deep learning models are increasingly popular in many machine learning applications where the training data may contain sensitive information. To provide formal and rigorous privac…
Private Stochastic Non-Convex Optimization: Adaptive Algorithms and Tighter Generalization Bounds
Yingxue Zhou, Xiangyi Chen, Mingyi Hong +2
We study differentially private (DP) algorithms for stochastic non-convex optimization. In this problem, the goal is to minimize the population loss over a -dimensional space gi…
ZO-AdaMM: Zeroth-Order Adaptive Momentum Method for Black-Box Optimization
Xiangyi Chen, Sijia Liu, Kaidi Xu +4
The adaptive momentum method (AdaMM), which uses past gradients to update descent directions and learning rates simultaneously, has become one of the most popular first-order optim…