3k citations · 7.9k across the 23 of their papers we have counts for
26 papers
Optimizing the Communication-Accuracy Trade-off in Federated Learning with Rate-Distortion Theory
Nicole Mitchell, Johannes Ballé, Zachary Charles +1
A significant bottleneck in federated learning (FL) is the network communication cost of sending model updates from client devices to the central server. We present a comprehensive…
A Field Guide to Federated Optimization
Jianyu Wang, Zachary Charles, Zheng Xu +50
Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy prote…
Convergence and Accuracy Trade-Offs in Federated Learning and Meta-Learning
Zachary Charles, Jakub Konečný
We study a family of algorithms, which we refer to as local update methods, generalizing many federated and meta-learning algorithms. We prove that for quadratic models, local upda…
On the Outsized Importance of Learning Rates in Local Update Methods
Zachary Charles, Jakub Konečný
We study a family of algorithms, which we refer to as local update methods, that generalize many federated learning and meta-learning algorithms. We prove that for quadratic object…
Adaptive Federated Optimization
Sashank Reddi, Zachary Charles, Manzil Zaheer +5
Federated learning is a distributed machine learning paradigm in which a large number of clients coordinate with a central server to learn a model without sharing their own trainin…
Advances and Open Problems in Federated Learning
Peter Kairouz, H. Brendan McMahan, Brendan Avent +56
Federated learning (FL) is a machine learning setting where many clients (e.g. mobile devices or whole organizations) collaboratively train a model under the orchestration of a cen…
Federated Learning with Autotuned Communication-Efficient Secure Aggregation
Keith Bonawitz, Fariborz Salehi, Jakub Konečný +2
Federated Learning enables mobile devices to collaboratively learn a shared inference model while keeping all the training data on a user's device, decoupling the ability to do mac…
Improving Federated Learning Personalization via Model Agnostic Meta Learning
Yihan Jiang, Jakub Konečný, Keith Rush +1
Federated Learning (FL) refers to learning a high quality global model based on decentralized data storage, without ever copying the raw data. A natural scenario arises with data c…
MLSys: The New Frontier of Machine Learning Systems
Alexander Ratner, Dan Alistarh, Gustavo Alonso +66
Machine learning (ML) techniques are enjoying rapidly increasing adoption. However, designing and implementing the systems that support ML models in real-world deployments remains…
Towards Federated Learning at Scale: System Design
Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp +11
Federated Learning is a distributed machine learning approach which enables model training on a large corpus of decentralized data. We have built a scalable production system for F…
A Privacy Preserving Randomized Gossip Algorithm via Controlled Noise Insertion
Filip Hanzely, Jakub Konečný, Nicolas Loizou +2
In this work we present a randomized gossip algorithm for solving the average consensus problem while at the same time protecting the information about the initial private values s…
Expanding the Reach of Federated Learning by Reducing Client Resource Requirements
Sebastian Caldas, Jakub Konečny, H. Brendan McMahan +1
Communication on heterogeneous edge networks is a fundamental bottleneck in Federated Learning (FL), restricting both model capacity and user participation. To address this issue,…
LEAF: A Benchmark for Federated Settings
Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu +5
Modern federated networks, such as those comprised of wearable devices, mobile phones, or autonomous vehicles, generate massive amounts of data each day. This wealth of data can he…
Stochastic, Distributed and Federated Optimization for Machine Learning
Jakub Konečný
We study optimization algorithms for the finite sum problems frequently arising in machine learning applications. First, we propose novel variants of stochastic gradient descent wi…
Randomized Distributed Mean Estimation: Accuracy vs Communication
Jakub Konečný, Peter Richtárik
We consider the problem of estimating the arithmetic average of a finite collection of real vectors stored in a distributed fashion across several compute nodes subject to a commun…
Federated Optimization: Distributed Machine Learning for On-Device Intelligence
Jakub Konečný, H. Brendan McMahan, Daniel Ramage +1
We introduce a new and increasingly relevant setting for distributed optimization in machine learning, where the data defining the optimization are unevenly distributed over an ext…
Federated Learning: Strategies for Improving Communication Efficiency
Jakub Konečný, H. Brendan McMahan, Felix X. Yu +3
Federated Learning is a machine learning setting where the goal is to train a high-quality centralized model while training data remains distributed over a large number of clients…
AIDE: Fast and Communication Efficient Distributed Optimization
Sashank J. Reddi, Jakub Konečný, Peter Richtárik +2
In this paper, we present two new communication-efficient methods for distributed minimization of an average of functions. The first algorithm is an inexact variant of the DANE alg…
Distributed Optimization with Arbitrary Local Solvers
Chenxin Ma, Jakub Konečný, Martin Jaggi +4
With the growth of data and necessity for distributed optimization methods, solvers that work well on a single machine must be re-designed to leverage distributed computation. Rece…
Federated Optimization:Distributed Optimization Beyond the Datacenter
Jakub Konečný, Brendan McMahan, Daniel Ramage
We introduce a new and increasingly relevant setting for distributed optimization in machine learning, where the data defining the optimization are distributed (unevenly) over an e…
Stop Wasting My Gradients: Practical SVRG
Reza Babanezhad, Mohamed Osama Ahmed, Alim Virani +3
We present and analyze several strategies for improving the performance of stochastic variance-reduced gradient (SVRG) methods. We first show that the convergence rate of these met…
Mini-Batch Semi-Stochastic Gradient Descent in the Proximal Setting
Jakub Konečný, Jie Liu, Peter Richtárik +1
We propose mS2GD: a method incorporating a mini-batching scheme for improving the theoretical complexity and practical performance of semi-stochastic gradient descent (S2GD). We co…
Semi-Stochastic Coordinate Descent
Jakub Konečný, Zheng Qu, Peter Richtárik
We propose a novel stochastic gradient method---semi-stochastic coordinate descent (S2CD)---for the problem of minimizing a strongly convex function represented as the average of a…
mS2GD: Mini-Batch Semi-Stochastic Gradient Descent in the Proximal Setting
Jakub Konečný, Jie Liu, Peter Richtárik +1
We propose a mini-batching scheme for improving the theoretical complexity and practical performance of semi-stochastic gradient descent applied to the problem of minimizing a stro…