198 citations · 764 across the 28 of their papers we have counts for
20 papers · 1 filter
Leveraging Function Space Aggregation for Federated Learning at Scale
Nikita Dhawan, Nicole Mitchell, Zachary Charles +2
The federated learning paradigm has motivated the development of methods for aggregating multiple client updates into a global server model, without sharing client data. Many feder…
Towards Federated Foundation Models: Scalable Dataset Pipelines for Group-Structured Learning
Zachary Charles, Nicole Mitchell, Krishna Pillutla +2
We introduce Dataset Grouper, a library to create large-scale group-structured (e.g., federated) datasets, enabling federated learning simulation at the scale of foundation models.…
Gradient Descent with Linearly Correlated Noise: Theory and Applications to Differential Privacy
Anastasia Koloskova, Ryan McKenna, Zachary Charles +2
We study gradient descent under linearly correlated noise. Our work is motivated by recent practical methods for optimization with differential privacy (DP), such as DP-FTRL, which…
Federated Automatic Differentiation
Keith Rush, Zachary Charles, Zachary Garrett
Federated learning (FL) is a general framework for learning across an axis of group partitioned data (heterogeneous clients) while preserving data privacy, under the orchestration…
Federated Select: A Primitive for Communication- and Memory-Efficient Federated Learning
Zachary Charles, Kallista Bonawitz, Stanislav Chiknavaryan +2
Federated learning (FL) is a framework for machine learning across heterogeneous client devices in a privacy-preserving fashion. To date, most FL algorithms learn a "global" server…
Motley: Benchmarking Heterogeneity and Personalization in Federated Learning
Shanshan Wu, Tian Li, Zachary Charles +4
Personalized federated learning considers learning models unique to each client in a heterogeneous network. The resulting client-specific models have been purported to improve metr…
Optimizing the Communication-Accuracy Trade-off in Federated Learning with Rate-Distortion Theory
Nicole Mitchell, Johannes Ballé, Zachary Charles +1
A significant bottleneck in federated learning (FL) is the network communication cost of sending model updates from client devices to the central server. We present a comprehensive…
A Field Guide to Federated Optimization
Jianyu Wang, Zachary Charles, Zheng Xu +50
Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy prote…
On Large-Cohort Training for Federated Learning
Zachary Charles, Zachary Garrett, Zhouyuan Huo +2
Federated learning methods typically learn a model by iteratively sampling updates from a population of clients. In this work, we explore how the number of clients sampled at each…
Local Adaptivity in Federated Learning: Convergence and Consistency
Jianyu Wang, Zheng Xu, Zachary Garrett +3
The federated learning (FL) framework trains a machine learning model using decentralized data stored at edge client devices by periodically aggregating locally trained models. Pop…
Convergence and Accuracy Trade-Offs in Federated Learning and Meta-Learning
Zachary Charles, Jakub Konečný
We study a family of algorithms, which we refer to as local update methods, generalizing many federated and meta-learning algorithms. We prove that for quadratic models, local upda…
On the Outsized Importance of Learning Rates in Local Update Methods
Zachary Charles, Jakub Konečný
We study a family of algorithms, which we refer to as local update methods, that generalize many federated learning and meta-learning algorithms. We prove that for quadratic object…
Adaptive Federated Optimization
Sashank Reddi, Zachary Charles, Manzil Zaheer +5
Federated learning is a distributed machine learning paradigm in which a large number of clients coordinate with a central server to learn a model without sharing their own trainin…
Advances and Open Problems in Federated Learning
Peter Kairouz, H. Brendan McMahan, Brendan Avent +56
Federated learning (FL) is a machine learning setting where many clients (e.g. mobile devices or whole organizations) collaboratively train a model under the orchestration of a cen…
Improving the convergence of SGD through adaptive batch sizes
Scott Sievert, Shrey Shah
Mini-batch stochastic gradient descent (SGD) and variants thereof approximate the objective function's gradient with a small number of training examples, aka the batch size. Small…
DETOX: A Redundancy-based Framework for Faster and More Robust Gradient Aggregation
Shashank Rajput, Hongyi Wang, Zachary Charles +1
To improve the resilience of distributed training to worst-case, or Byzantine node failures, several recent approaches have replaced gradient averaging with robust aggregation meth…
Convergence and Margin of Adversarial Training on Separable Data
Zachary Charles, Shashank Rajput, Stephen Wright +1
Adversarial training is a technique for training robust machine learning models. To encourage robustness, it iteratively computes adversarial examples for the model, and then re-tr…
Does Data Augmentation Lead to Positive Margin?
Shashank Rajput, Zhili Feng, Zachary Charles +2
Data augmentation (DA) is commonly used during model training, as it significantly improves test error and model robustness. DA artificially expands the training set by applying ra…
ErasureHead: Distributed Gradient Descent without Delays Using Approximate Gradient Coding
Hongyi Wang, Zachary Charles, Dimitris Papailiopoulos
We present ErasureHead, a new approach for distributed gradient descent (GD) that mitigates system delays by employing approximate gradient coding. Gradient coded distributed GD us…
A Geometric Perspective on the Transferability of Adversarial Directions
Zachary Charles, Harrison Rosenberg, Dimitris Papailiopoulos
State-of-the-art machine learning models frequently misclassify inputs that have been perturbed in an adversarial manner. Adversarial perturbations generated for a given input and…