Ditto: Fair and Robust Federated Learning Through Personalization

Abstract

Fairness and robustness are two important concerns for federated learning systems. In this work, we identify that robustness to data and model poisoning attacks and fairness, measured as the uniformity of performance across devices, are competing constraints in statistically heterogeneous networks. To address these constraints, we propose employing a simple, general framework for personalized federated learning, Ditto, that can inherently provide fairness and robustness benefits, and develop a scalable solver for it. Theoretically, we analyze the ability of Ditto to achieve fairness and robustness simultaneously on a class of linear problems. Empirically, across a suite of federated datasets, we show that Ditto not only achieves competitive performance relative to recent personalization methods, but also enables more accurate, robust, and fair models relative to state-of-the-art fair or robust baselines.

Accepted by ICML 2021

References in corpus (36)

• Overcoming catastrophic forgetting in neural networks
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning with Non-IID Data
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Robust Aggregation for Federated Learning
• How To Backdoor Federated Learning
• Poisoning Attacks against Support Vector Machines
• Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
• EMNIST: an extension of MNIST to handwritten letters
• Can You Really Backdoor Federated Learning?
• Improving Federated Learning Personalization via Model Agnostic Meta Learning
• Personalized Federated Learning: A Meta-Learning Approach
• Adaptive Personalized Federated Learning
• Three Approaches for Personalization with Applications to Federated Learning
• Progress & Compress: A scalable framework for continual learning
• LEAF: A Benchmark for Federated Settings
• Federated Meta-Learning with Fast Convergence and Efficient Communication
• Agnostic Federated Learning
• Federated Learning of a Mixture of Global and Local Models
• Personalized Federated Learning with Moreau Envelopes
• Fairness Without Demographics in Repeated Loss Minimization
• Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks
• Think Locally, Act Globally: Federated Learning with Local and Global Representations
• MetaPoison: Practical General-purpose Clean-label Data Poisoning
• Salvaging Federated Learning by Local Adaptation
• Lower Bounds and Optimal Algorithms for Personalized Federated Learning
• Hierarchically Fair Federated Learning
• A Reputation Mechanism Is All You Need: Collaborative Fairness and Adversarial Robustness in Federated Learning
• On Adversarial Bias and the Robustness of Fair Machine Learning
• Federated Learning Meets Multi-objective Optimization
• Federated Reconstruction: Partially Local Federated Learning
• Distributionally Robust Federated Averaging
• Federated Residual Learning
• Data Poisoning Attacks on Federated Machine Learning

Cited by in corpus (23)

• FedALA: Adaptive Local Aggregation for Personalized Federated Learning
• Flexible Clustered Federated Learning for Client-Level Data Distribution Shift
• FedCP: Separating Feature Information for Personalized Federated Learning via Conditional Policy
• An Experimental Study of Byzantine-Robust Aggregation Schemes in Federated Learning
• A review of federated learning in renewable energy applications: Potential, challenges, and future directions
• Exploiting Shared Representations for Personalized Federated Learning
• The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning
• Auto-weighted Robust Federated Learning with Corrupted Data Sources
• Fed-ensemble: Improving Generalization through Model Ensembling in Federated Learning
• Federated learning-outcome prediction with multi-layer privacy protection
• Communication-Efficient Robust Federated Learning with Noisy Labels
• Connecting Low-Loss Subspace for Personalized Federated Learning
• Implicit Model Specialization through DAG-based Decentralized Federated Learning
• CDKT-FL: Cross-Device Knowledge Transfer using Proxy Dataset in Federated Learning
• Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing
• Joint Local Relational Augmentation and Global Nash Equilibrium for Federated Learning with Non-IID Data
• Auxo: Efficient Federated Learning via Scalable Client Clustering
• Personalized Federated Learning: A Unified Framework and Universal Optimization Techniques
• Towards Fast and Stable Federated Learning: Confronting Heterogeneity via Knowledge Anchor
• Uncertainty-Aware Label Refinement on Hypergraphs for Personalized Federated Facial Expression Recognition
• FedPrune: Towards Inclusive Federated Learning
• Federated Learning with Fair Averaging
• BOBA: Byzantine-Robust Federated Learning with Label Skewness