3k citations · 7.9k across the 23 of their papers we have counts for
19 papers · 1 filter
Optimizing the Communication-Accuracy Trade-off in Federated Learning with Rate-Distortion Theory
Nicole Mitchell, Johannes Ballé, Zachary Charles +1
A significant bottleneck in federated learning (FL) is the network communication cost of sending model updates from client devices to the central server. We present a comprehensive…
A Field Guide to Federated Optimization
Jianyu Wang, Zachary Charles, Zheng Xu +50
Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy prote…
Convergence and Accuracy Trade-Offs in Federated Learning and Meta-Learning
Zachary Charles, Jakub Konečný
We study a family of algorithms, which we refer to as local update methods, generalizing many federated and meta-learning algorithms. We prove that for quadratic models, local upda…
On the Outsized Importance of Learning Rates in Local Update Methods
Zachary Charles, Jakub Konečný
We study a family of algorithms, which we refer to as local update methods, that generalize many federated learning and meta-learning algorithms. We prove that for quadratic object…
Adaptive Federated Optimization
Sashank Reddi, Zachary Charles, Manzil Zaheer +5
Federated learning is a distributed machine learning paradigm in which a large number of clients coordinate with a central server to learn a model without sharing their own trainin…
Advances and Open Problems in Federated Learning
Peter Kairouz, H. Brendan McMahan, Brendan Avent +56
Federated learning (FL) is a machine learning setting where many clients (e.g. mobile devices or whole organizations) collaboratively train a model under the orchestration of a cen…
Improving Federated Learning Personalization via Model Agnostic Meta Learning
Yihan Jiang, Jakub Konečný, Keith Rush +1
Federated Learning (FL) refers to learning a high quality global model based on decentralized data storage, without ever copying the raw data. A natural scenario arises with data c…
MLSys: The New Frontier of Machine Learning Systems
Alexander Ratner, Dan Alistarh, Gustavo Alonso +66
Machine learning (ML) techniques are enjoying rapidly increasing adoption. However, designing and implementing the systems that support ML models in real-world deployments remains…
Towards Federated Learning at Scale: System Design
Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp +11
Federated Learning is a distributed machine learning approach which enables model training on a large corpus of decentralized data. We have built a scalable production system for F…
Expanding the Reach of Federated Learning by Reducing Client Resource Requirements
Sebastian Caldas, Jakub Konečny, H. Brendan McMahan +1
Communication on heterogeneous edge networks is a fundamental bottleneck in Federated Learning (FL), restricting both model capacity and user participation. To address this issue,…
LEAF: A Benchmark for Federated Settings
Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu +5
Modern federated networks, such as those comprised of wearable devices, mobile phones, or autonomous vehicles, generate massive amounts of data each day. This wealth of data can he…
Stochastic, Distributed and Federated Optimization for Machine Learning
Jakub Konečný
We study optimization algorithms for the finite sum problems frequently arising in machine learning applications. First, we propose novel variants of stochastic gradient descent wi…
Federated Optimization: Distributed Machine Learning for On-Device Intelligence
Jakub Konečný, H. Brendan McMahan, Daniel Ramage +1
We introduce a new and increasingly relevant setting for distributed optimization in machine learning, where the data defining the optimization are unevenly distributed over an ext…
Federated Learning: Strategies for Improving Communication Efficiency
Jakub Konečný, H. Brendan McMahan, Felix X. Yu +3
Federated Learning is a machine learning setting where the goal is to train a high-quality centralized model while training data remains distributed over a large number of clients…
Distributed Optimization with Arbitrary Local Solvers
Chenxin Ma, Jakub Konečný, Martin Jaggi +4
With the growth of data and necessity for distributed optimization methods, solvers that work well on a single machine must be re-designed to leverage distributed computation. Rece…
Federated Optimization:Distributed Optimization Beyond the Datacenter
Jakub Konečný, Brendan McMahan, Daniel Ramage
We introduce a new and increasingly relevant setting for distributed optimization in machine learning, where the data defining the optimization are distributed (unevenly) over an e…
Stop Wasting My Gradients: Practical SVRG
Reza Babanezhad, Mohamed Osama Ahmed, Alim Virani +3
We present and analyze several strategies for improving the performance of stochastic variance-reduced gradient (SVRG) methods. We first show that the convergence rate of these met…
Mini-Batch Semi-Stochastic Gradient Descent in the Proximal Setting
Jakub Konečný, Jie Liu, Peter Richtárik +1
We propose mS2GD: a method incorporating a mini-batching scheme for improving the theoretical complexity and practical performance of semi-stochastic gradient descent (S2GD). We co…
mS2GD: Mini-Batch Semi-Stochastic Gradient Descent in the Proximal Setting
Jakub Konečný, Jie Liu, Peter Richtárik +1
We propose a mini-batching scheme for improving the theoretical complexity and practical performance of semi-stochastic gradient descent applied to the problem of minimizing a stro…