Agnostic Federated Learning

Abstract

A key learning scenario in large-scale applications is that of federated learning, where a centralized model is trained based on data originating from a large number of clients. We argue that, with the existing training and inference, federated models can be biased towards different clients. Instead, we propose a new framework of agnostic federated learning, where the centralized model is optimized for any target distribution formed by a mixture of the client distributions. We further show that this framework naturally yields a notion of fairness. We present data-dependent Rademacher complexity guarantees for learning with this objective, which guide the definition of an algorithm for agnostic federated learning. We also give a fast stochastic optimization algorithm for solving the corresponding optimization problem, for which we prove convergence bounds, assuming a convex loss function and hypothesis set. We further empirically demonstrate the benefits of our approach in several datasets. Beyond federated learning, our framework and algorithm can be of interest to other learning scenarios such as cloud computing, domain adaptation, drifting, and other contexts where the training and test distributions do not coincide.

30 pages

References in corpus (3)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• Chameleons in imagined conversations: A new approach to understanding coordination of linguistic style in dialogs

Cited by in corpus (14)

• On the Convergence of Local Descent Methods in Federated Learning
• Fairness and Accuracy in Federated Learning
• Multi-institutional Collaborations for Improving Deep Learning-based Magnetic Resonance Image Reconstruction Using Federated Learning
• From Federated Learning to Federated Neural Architecture Search: A Survey
• Federated Mixture of Experts
• Federated learning with class imbalance reduction
• Bandwidth Allocation for Multiple Federated Learning Services in Wireless Edge Networks
• Federated Block Coordinate Descent Scheme for Learning Global and Personalized Models
• Accurate and Fast Federated Learning via Combinatorial Multi-Armed Bandits
• Linear Regression Games: Convergence Guarantees to Approximate Out-of-Distribution Solutions
• Federated Learning for Open Banking
• Learning-based Prediction, Rendering and Transmission for Interactive Virtual Reality in RIS-Assisted Terahertz Networks
• On Addressing Heterogeneity in Federated Learning for Autonomous Vehicles Connected to a Drone Orchestrator
• AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learning