Agnostic Federated Learning

Abstract

A key learning scenario in large-scale applications is that of federated learning, where a centralized model is trained based on data originating from a large number of clients. We argue that, with the existing training and inference, federated models can be biased towards different clients. Instead, we propose a new framework of agnostic federated learning, where the centralized model is optimized for any target distribution formed by a mixture of the client distributions. We further show that this framework naturally yields a notion of fairness. We present data-dependent Rademacher complexity guarantees for learning with this objective, which guide the definition of an algorithm for agnostic federated learning. We also give a fast stochastic optimization algorithm for solving the corresponding optimization problem, for which we prove convergence bounds, assuming a convex loss function and hypothesis set. We further empirically demonstrate the benefits of our approach in several datasets. Beyond federated learning, our framework and algorithm can be of interest to other learning scenarios such as cloud computing, domain adaptation, drifting, and other contexts where the training and test distributions do not coincide.

30 pages

References in corpus (9)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Federated Learning: Strategies for Improving Communication Efficiency
• Federated Optimization: Distributed Machine Learning for On-Device Intelligence
• Federated Learning for Mobile Keyboard Prediction
• Chameleons in imagined conversations: A new approach to understanding coordination of linguistic style in dialogs
• Projection onto the probability simplex: An efficient algorithm with a simple proof, and an application
• Distributed Mean Estimation with Limited Communication
• Federated Multi-Task Learning
• Contextual Language Model Adaptation for Conversational Agents

Cited by in corpus (107)

• A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
• Robust Aggregation for Federated Learning
• SCAFFOLD: Stochastic Controlled Averaging for Federated Learning
• Ensemble Distillation for Robust Model Fusion in Federated Learning
• From Distributed Machine Learning to Federated Learning: A Survey
• FedML: A Research Library and Benchmark for Federated Machine Learning
• Fair Resource Allocation in Federated Learning
• Adaptive Personalized Federated Learning
• Three Approaches for Personalization with Applications to Federated Learning
• Ditto: Fair and Robust Federated Learning Through Personalization
• Personalized Federated Learning with Moreau Envelopes
• Federated Learning: A Signal Processing Perspective
• SecureBoost: A Lossless Federated Learning Framework
• On the Convergence of Local Descent Methods in Federated Learning
• Federated Learning in Mobile Edge Networks: A Comprehensive Survey
• Think Locally, Act Globally: Federated Learning with Local and Global Representations
• FedGroup: Efficient Clustered Federated Learning via Decomposed Data-Driven Measure
• Fusion of Probability Density Functions
• Flexible Clustered Federated Learning for Client-Level Data Distribution Shift
• Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning
• Federated Multi-Task Learning under a Mixture of Distributions
• VAFL: a Method of Vertical Asynchronous Federated Learning
• Federated Learning on Non-IID Data Silos: An Experimental Study
• Dynamic Sampling and Selective Masking for Communication-Efficient Federated Learning
• Differential Privacy Has Disparate Impact on Model Accuracy
• FedSAE: A Novel Self-Adaptive Federated Learning Framework in Heterogeneous Systems
• Federated Learning for Healthcare Informatics
• The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning
• FedCluster: Boosting the Convergence of Federated Learning via Cluster-Cycling
• Federated Learning with Unbiased Gradient Aggregation and Controllable Meta Updating
• Oort: Efficient Federated Learning via Guided Participant Selection
• Fairness without Demographics through Adversarially Reweighted Learning
• Patchwork Learning: A Paradigm Towards Integrative Analysis across Diverse Biomedical Data Sources
• FedCV: A Federated Learning Framework for Diverse Computer Vision Tasks
• A Reputation Mechanism Is All You Need: Collaborative Fairness and Adversarial Robustness in Federated Learning
• Personalized and privacy-preserving federated heterogeneous medical image analysis with PPPML-HMI
• Privacy-preserving Federated Adversarial Domain Adaption over Feature Groups for Interpretability
• Fairness and Accuracy in Federated Learning
• Auto-weighted Robust Federated Learning with Corrupted Data Sources
• Improving Fairness via Federated Learning
• Invariant Risk Minimization Games
• Noise-tolerant fair classification
• Federated Learning Meets Multi-objective Optimization
• Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning
• Multi-site fMRI Analysis Using Privacy-preserving Federated Learning and Domain Adaptation: ABIDE Results
• Federated Fuzzy Neural Network with Evolutionary Rule Learning
• Task-Robust Model-Agnostic Meta-Learning
• Communication-Efficient Robust Federated Learning with Noisy Labels
• An Optimal Transport Approach to Personalized Federated Learning
• Fairness without the sensitive attribute via Causal Variational Autoencoder
• BAFFLE : Blockchain Based Aggregator Free Federated Learning
• FedH2L: Federated Learning with Model and Statistical Heterogeneity
• Personalized Federated Learning through Local Memorization
• Heterogeneity for the Win: One-Shot Federated Clustering
• FedEval: A Holistic Evaluation Framework for Federated Learning
• Training Fair Models in Federated Learning without Data Privacy Infringement
• Federated Residual Learning
• A Single-Loop Smoothed Gradient Descent-Ascent Algorithm for Nonconvex-Concave Min-Max Problems
• Federated Learning with Superquantile Aggregation for Heterogeneous Data
• Device Heterogeneity in Federated Learning: A Superquantile Approach
• Multi-institutional Collaborations for Improving Deep Learning-based Magnetic Resonance Image Reconstruction Using Federated Learning
• What Do We Mean by Generalization in Federated Learning?
• Achieving Model Fairness in Vertical Federated Learning
• Newton-type Methods for Minimax Optimization
• Federated Composite Optimization
• From Federated Learning to Federated Neural Architecture Search: A Survey
• On Tilted Losses in Machine Learning: Theory and Applications
• Byzantine-Resilient Secure Federated Learning
• Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspective
• Characterizing Impacts of Heterogeneity in Federated Learning upon Large-Scale Smartphone Data
• Federated Mixture of Experts
• Federated learning with class imbalance reduction
• Toward an Automated Auction Framework for Wireless Federated Learning Services Market
• Wireless for Machine Learning
• Unified Group Fairness on Federated Learning
• Fairness-aware Agnostic Federated Learning
• Coping with Label Shift via Distributionally Robust Optimisation
• A Theory of Multiple-Source Adaptation with Limited Target Labeled Data
• Bandwidth Allocation for Multiple Federated Learning Services in Wireless Edge Networks
• A Discriminative Technique for Multiple-Source Adaptation
• Anarchic Federated Learning
• Federated Block Coordinate Descent Scheme for Learning Global and Personalized Models
• Voting-based Approaches For Differentially Private Federated Learning
• Accurate and Fast Federated Learning via Combinatorial Multi-Armed Bandits
• Rate-Distortion Theoretic Bounds on Generalization Error for Distributed Learning
• Tilted Empirical Risk Minimization
• Linear Regression Games: Convergence Guarantees to Approximate Out-of-Distribution Solutions
• Learning-based Prediction, Rendering and Transmission for Interactive Virtual Reality in RIS-Assisted Terahertz Networks
• Federated Learning for Open Banking
• Performance Analysis and Characterization of Training Deep Learning Models on Mobile Devices
• Communication-Efficient Agnostic Federated Averaging
• CDMA: A Practical Cross-Device Federated Learning Algorithm for General Minimax Problems
• An Operator Splitting View of Federated Learning
• Adversarial Robustness through Bias Variance Decomposition: A New Perspective for Federated Learning
• FedProf: Selective Federated Learning with Representation Profiling
• Probabilistic Federated Learning of Neural Networks Incorporated with Global Posterior Information
• Unsupervised Learning of Debiased Representations with Pseudo-Attributes
• Optimising cost vs accuracy of decentralised analytics in fog computing environments
• Fed-EINI: An Efficient and Interpretable Inference Framework for Decision Tree Ensembles in Federated Learning
• Semi-Anchored Multi-Step Gradient Descent Ascent Method for Structured Nonconvex-Nonconcave Composite Minimax Problems
• Multi-task Federated Edge Learning (MtFEEL) in Wireless Networks
• Distributionally Robust Optimization with Moment Ambiguity Sets
• Compositional federated learning: Applications in distributionally robust averaging and meta learning
• FLBench: A Benchmark Suite for Federated Learning
• Linear Speedup in Personalized Collaborative Learning
• AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learning
• On Addressing Heterogeneity in Federated Learning for Autonomous Vehicles Connected to a Drone Orchestrator