605 citations · 1.2k across the 29 of their papers we have counts for
21 papers · 1 filter
(Amplified) Banded Matrix Factorization: A unified approach to private training
Christopher A. Choquette-Choo, Arun Ganesh, Ryan McKenna +4
Matrix factorization (MF) mechanisms for differential privacy (DP) have substantially improved the state-of-the-art in privacy-utility-computation tradeoffs for ML applications in…
Federated Learning of Gboard Language Models with Differential Privacy
Zheng Xu, Yanxiang Zhang, Galen Andrew +5
We train language models (LMs) with federated learning (FL) and differential privacy (DP) in the Google Keyboard (Gboard). We apply the DP-Follow-the-Regularized-Leader (DP-FTRL)~\…
Can Public Large Language Models Help Private Cross-device Federated Learning?
Boxin Wang, Yibo Jacky Zhang, Yuan Cao +5
We study (differentially) private federated learning (FL) of language models. The language models in cross-device FL are relatively small, which can be trained with meaningful form…
An Empirical Evaluation of Federated Contextual Bandit Algorithms
Alekh Agarwal, H. Brendan McMahan, Zheng Xu
As the adoption of federated learning increases for learning from sensitive data local to user devices, it is natural to ask if the learning can be done using implicit signals gene…
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy
Natalia Ponomareva, Hussein Hazimeh, Alex Kurakin +6
ML models are ubiquitous in real world applications and are a constant focus of research. At the same time, the community has started to realize the importance of protecting the pr…
On the Convergence of Federated Averaging with Cyclic Client Participation
Yae Jee Cho, Pranay Sharma, Gauri Joshi +3
Federated Averaging (FedAvg) and its variants are the most popular optimization algorithms in federated learning (FL). Previous convergence analyses of FedAvg either assume full cl…
Learning to Generate Image Embeddings with User-level Differential Privacy
Zheng Xu, Maxwell Collins, Yuxiao Wang +6
Small on-device models have been successfully trained with user-level differential privacy (DP) for next word prediction and image classification tasks in the past. However, existi…
On the Unreasonable Effectiveness of Federated Averaging with Heterogeneous Data
Jianyu Wang, Rudrajit Das, Gauri Joshi +3
Existing theory predicts that data heterogeneity will degrade the performance of the Federated Averaging (FedAvg) algorithm in federated learning. However, in practice, the simple…
Beyond Uniform Lipschitz Condition in Differentially Private Optimization
Rudrajit Das, Satyen Kale, Zheng Xu +2
Most prior results on differentially private stochastic gradient descent (DP-SGD) are derived under the simplistic assumption of uniform Lipschitzness, i.e., the per-sample gradien…
Motley: Benchmarking Heterogeneity and Personalization in Federated Learning
Shanshan Wu, Tian Li, Zachary Charles +4
Personalized federated learning considers learning models unique to each client in a heterogeneous network. The resulting client-specific models have been purported to improve metr…
Exploring Model Robustness with Adaptive Networks and Improved Adversarial Training
Zheng Xu, Ali Shafahi, Tom Goldstein
Adversarial training has proven to be effective in hardening networks against adversarial examples. However, the gained robustness is limited by network capacity and number of trai…
Advances and Open Problems in Federated Learning
Peter Kairouz, H. Brendan McMahan, Brendan Avent +56
Federated learning (FL) is a machine learning setting where many clients (e.g. mobile devices or whole organizations) collaboratively train a model under the orchestration of a cen…
Adversarial Training for Free!
Ali Shafahi, Mahyar Najibi, Amin Ghiasi +6
Adversarial training, in which a network is trained on adversarial examples, is one of the few defenses against adversarial attacks that withstands strong attacks. Unfortunately, t…
The Impact of Neural Network Overparameterization on Gradient Confusion and Stochastic Gradient Descent
Karthik A. Sankararaman, Soham De, Zheng Xu +2
This paper studies how neural network architecture affects the speed of training. We introduce a simple concept called gradient confusion to help formally analyze this. When gradie…
Visualizing the Loss Landscape of Neural Nets
Hao Li, Zheng Xu, Gavin Taylor +2
Neural network training relies on our ability to find "good" minimizers of highly non-convex loss functions. It is well-known that certain network architecture designs (e.g., skip…
Group-driven Reinforcement Learning for Personalized mHealth Intervention
Feiyun Zhu, Jun Guo, Zheng Xu +2
Due to the popularity of smartphones and wearable devices nowadays, mobile health (mHealth) technologies are promising to bring positive and wide impacts on people's health. State-…
Adaptive Consensus ADMM for Distributed Optimization
Zheng Xu, Gavin Taylor, Hao Li +3
The alternating direction method of multipliers (ADMM) is commonly used for distributed model fitting problems, but its performance and reliability depend strongly on user-defined…
Training Quantized Nets: A Deeper Understanding
Hao Li, Soham De, Zheng Xu +3
Currently, deep neural networks are deployed on low-power portable devices by first training a full-precision model using powerful hardware, and then deriving a corresponding low-p…
Stabilizing Adversarial Nets With Prediction Methods
Abhay Yadav, Sohil Shah, Zheng Xu +2
Adversarial neural networks solve many important problems in data science, but are notoriously difficult to train. These difficulties come from the fact that optimal weights for ad…
Training Neural Networks Without Gradients: A Scalable ADMM Approach
Gavin Taylor, Ryan Burmeister, Zheng Xu +3
With the growing importance of large network models and enormous training datasets, GPUs have become increasingly necessary to train neural networks. This is largely because conven…
Adaptive ADMM with Spectral Penalty Parameter Selection
Zheng Xu, Mario A. T. Figueiredo, Tom Goldstein
The alternating direction method of multipliers (ADMM) is a versatile tool for solving a wide range of constrained optimization problems, with differentiable or non-differentiable…