605 citations · 1.2k across the 29 of their papers we have counts for
33 papers
Private Federated Learning in Gboard
Yuanbo Zhang, Daniel Ramage, Zheng Xu +3
This white paper describes recent advances in Gboard(Google Keyboard)'s use of federated learning, DP-Follow-the-Regularized-Leader (DP-FTRL) algorithm, and secure aggregation tech…
(Amplified) Banded Matrix Factorization: A unified approach to private training
Christopher A. Choquette-Choo, Arun Ganesh, Ryan McKenna +4
Matrix factorization (MF) mechanisms for differential privacy (DP) have substantially improved the state-of-the-art in privacy-utility-computation tradeoffs for ML applications in…
Federated Learning of Gboard Language Models with Differential Privacy
Zheng Xu, Yanxiang Zhang, Galen Andrew +5
We train language models (LMs) with federated learning (FL) and differential privacy (DP) in the Google Keyboard (Gboard). We apply the DP-Follow-the-Regularized-Leader (DP-FTRL)~\…
Can Public Large Language Models Help Private Cross-device Federated Learning?
Boxin Wang, Yibo Jacky Zhang, Yuan Cao +5
We study (differentially) private federated learning (FL) of language models. The language models in cross-device FL are relatively small, which can be trained with meaningful form…
An Empirical Evaluation of Federated Contextual Bandit Algorithms
Alekh Agarwal, H. Brendan McMahan, Zheng Xu
As the adoption of federated learning increases for learning from sensitive data local to user devices, it is natural to ask if the learning can be done using implicit signals gene…
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy
Natalia Ponomareva, Hussein Hazimeh, Alex Kurakin +6
ML models are ubiquitous in real world applications and are a constant focus of research. At the same time, the community has started to realize the importance of protecting the pr…
On the Convergence of Federated Averaging with Cyclic Client Participation
Yae Jee Cho, Pranay Sharma, Gauri Joshi +3
Federated Averaging (FedAvg) and its variants are the most popular optimization algorithms in federated learning (FL). Previous convergence analyses of FedAvg either assume full cl…
Learning to Generate Image Embeddings with User-level Differential Privacy
Zheng Xu, Maxwell Collins, Yuxiao Wang +6
Small on-device models have been successfully trained with user-level differential privacy (DP) for next word prediction and image classification tasks in the past. However, existi…
Self-Aligned Concave Curve: Illumination Enhancement for Unsupervised Adaptation
Wenjing Wang, Zhengbo Xu, Haofeng Huang +1
Low light conditions not only degrade human visual experience, but also reduce the performance of downstream machine analytics. Although many works have been designed for low-light…
On the Unreasonable Effectiveness of Federated Averaging with Heterogeneous Data
Jianyu Wang, Rudrajit Das, Gauri Joshi +3
Existing theory predicts that data heterogeneity will degrade the performance of the Federated Averaging (FedAvg) algorithm in federated learning. However, in practice, the simple…
Beyond Uniform Lipschitz Condition in Differentially Private Optimization
Rudrajit Das, Satyen Kale, Zheng Xu +2
Most prior results on differentially private stochastic gradient descent (DP-SGD) are derived under the simplistic assumption of uniform Lipschitzness, i.e., the per-sample gradien…
Motley: Benchmarking Heterogeneity and Personalization in Federated Learning
Shanshan Wu, Tian Li, Zachary Charles +4
Personalized federated learning considers learning models unique to each client in a heterogeneous network. The resulting client-specific models have been purported to improve metr…
Digging into Primary Financial Market: Challenges and Opportunities of Adopting Blockchain
Ji Liu, Zheng Xu, Yanmei Zhang +3
Since the emergence of blockchain technology, its application in the financial market has always been an area of focus and exploration by all parties. With the characteristics of a…
Exploring Model Robustness with Adaptive Networks and Improved Adversarial Training
Zheng Xu, Ali Shafahi, Tom Goldstein
Adversarial training has proven to be effective in hardening networks against adversarial examples. However, the gained robustness is limited by network capacity and number of trai…
Advances and Open Problems in Federated Learning
Peter Kairouz, H. Brendan McMahan, Brendan Avent +56
Federated learning (FL) is a machine learning setting where many clients (e.g. mobile devices or whole organizations) collaboratively train a model under the orchestration of a cen…
Adversarial Training for Free!
Ali Shafahi, Mahyar Najibi, Amin Ghiasi +6
Adversarial training, in which a network is trained on adversarial examples, is one of the few defenses against adversarial attacks that withstands strong attacks. Unfortunately, t…
The Impact of Neural Network Overparameterization on Gradient Confusion and Stochastic Gradient Descent
Karthik A. Sankararaman, Soham De, Zheng Xu +2
This paper studies how neural network architecture affects the speed of training. We introduce a simple concept called gradient confusion to help formally analyze this. When gradie…
Universal Adversarial Training
Ali Shafahi, Mahyar Najibi, Zheng Xu +3
Standard adversarial attacks change the predicted class label of a selected image by adding specially tailored small perturbations to its pixels. In contrast, a universal perturbat…
Accurate and efficient video de-fencing using convolutional neural networks and temporal information
Chen Du, Byeongkeun Kang, Zheng Xu +2
De-fencing is to eliminate the captured fence on an image or a video, providing a clear view of the scene. It has been applied for many purposes including assisting photographers a…
The Effectiveness of Instance Normalization: a Strong Baseline for Single Image Dehazing
Zheng Xu, Xitong Yang, Xue Li +1
We propose a novel deep neural network architecture for the challenging problem of single image dehazing, which aims to recover the clear image from a degraded hazy image. Instead…
Learning from Multi-domain Artistic Images for Arbitrary Style Transfer
Zheng Xu, Michael Wilber, Chen Fang +2
We propose a fast feed-forward network for arbitrary style transfer, which can generate stylized image for previously unseen content and style image pairs. Besides the traditional…
Learning to Cluster for Proposal-Free Instance Segmentation
Yen-Chang Hsu, Zheng Xu, Zsolt Kira +1
This work proposed a novel learning objective to train a deep neural network to perform end-to-end image pixel clustering. We applied the approach to instance segmentation, which i…
Visualizing the Loss Landscape of Neural Nets
Hao Li, Zheng Xu, Gavin Taylor +2
Neural network training relies on our ability to find "good" minimizers of highly non-convex loss functions. It is well-known that certain network architecture designs (e.g., skip…
Group-driven Reinforcement Learning for Personalized mHealth Intervention
Feiyun Zhu, Jun Guo, Zheng Xu +2
Due to the popularity of smartphones and wearable devices nowadays, mobile health (mHealth) technologies are promising to bring positive and wide impacts on people's health. State-…