1.1k citations · 1.6k across the 34 of their papers we have counts for
14 papers · 1 filter
Learning from straggler clients in federated learning
Andrew Hard, Antonious M. Girgis, Ehsan Amid +4
How well do existing federated learning algorithms learn from client devices that return model updates with a significant time delay? Is it even possible to learn effectively from…
Unintended Memorization in Large ASR Models, and How to Mitigate It
Lun Wang, Om Thakkar, Rajiv Mathews
It is well-known that neural networks can unintentionally memorize their training examples, causing privacy concerns. However, auditing memorization in large non-auto-regressive au…
Heterogeneous Federated Learning Using Knowledge Codistillation
Jared Lichtarge, Ehsan Amid, Shankar Kumar +3
Federated Averaging, and many federated learning algorithm variants which build upon it, have a limitation: all clients must share the same model architecture. This results in unus…
Recycling Scraps: Improving Private Learning by Leveraging Intermediate Checkpoints
Virat Shejwalkar, Arun Ganesh, Rajiv Mathews +5
In this work, we focus on improving the accuracy-variance trade-off for state-of-the-art differentially private machine learning (DP ML) methods. First, we design a general framewo…
Mixed Federated Learning: Joint Decentralized and Centralized Learning
Sean Augenstein, Andrew Hard, Lin Ning +4
Federated learning (FL) enables learning from decentralized privacy-sensitive data, with computations on raw data confined to take place at edge clients. This paper introduces mixe…
Online Model Compression for Federated Learning with Large Models
Tien-Ju Yang, Yonghui Xiao, Giovanni Motta +3
This paper addresses the challenges of training large neural network models under federated learning settings: high on-device memory usage and communication cost. The proposed Onli…
Public Data-Assisted Mirror Descent for Private Model Training
Ehsan Amid, Arun Ganesh, Rajiv Mathews +6
In this paper, we revisit the problem of using in-distribution public data to improve the privacy/utility trade-offs for differentially private (DP) model training. (Here, public d…
Jointly Learning from Decentralized (Federated) and Centralized Data to Mitigate Distribution Shift
Sean Augenstein, Andrew Hard, Kurt Partridge +1
With privacy as a motivation, Federated Learning (FL) is an increasingly used paradigm where learning takes place collectively on edge devices, each with a cache of user-generated…
Revealing and Protecting Labels in Distributed Training
Trung Dang, Om Thakkar, Swaroop Ramaswamy +3
Distributed learning paradigms such as federated learning often involve transmission of model updates, or gradients, over a network, thereby avoiding transmission of private data.…
Communication-Efficient Agnostic Federated Averaging
Jae Ro, Mingqing Chen, Rajiv Mathews +2
In distributed learning settings such as federated learning, the training algorithm can be potentially biased towards different clients. Mohri et al. (2019) proposed a domain-agnos…
Training Production Language Models without Memorizing User Data
Swaroop Ramaswamy, Om Thakkar, Rajiv Mathews +3
This paper presents the first consumer-scale next-word prediction (NWP) model trained with Federated Learning (FL) while leveraging the Differentially Private Federated Averaging (…
Understanding Unintended Memorization in Federated Learning
Om Thakkar, Swaroop Ramaswamy, Rajiv Mathews +1
Recent works have shown that generative sequence models (e.g., language models) have a tendency to memorize rare or unique sequences in the training data. Since useful models are o…
Generative Models for Effective ML on Private, Decentralized Datasets
Sean Augenstein, H. Brendan McMahan, Daniel Ramage +5
To improve real-world applications of machine learning, experienced modelers develop intuition about their datasets, their models, and how the two interact. Manual inspection of ra…
Federated Evaluation of On-device Personalization
Kangkang Wang, Rajiv Mathews, Chloé Kiddon +3
Federated learning is a distributed, on-device computation framework that enables training global models without exporting sensitive user data to servers. In this work, we describe…