Federated Reconstruction: Partially Local Federated Learning
arXiv:2102.03448
Abstract
Personalization methods in federated learning aim to balance the benefits of federated and local training for data availability, communication cost, and robustness to client heterogeneity. Approaches that require clients to communicate all model parameters can be undesirable due to privacy and communication constraints. Other approaches require always-available or stateful clients, impractical in large-scale cross-device settings. We introduce Federated Reconstruction, the first model-agnostic framework for partially local federated learning suitable for training and inference at scale. We motivate the framework via a connection to model-agnostic meta learning, empirically demonstrate its performance over existing approaches for collaborative filtering and next word prediction, and release an open-source library for evaluating approaches in this setting. We also describe the successful deployment of this approach at scale for federated collaborative filtering in a mobile keyboard application.
35th Conference on Neural Information Processing Systems (NeurIPS 2021). Code: https://github.com/google-research/federated/tree/master/reconstruction
References in corpus (21)
- Towards Federated Learning at Scale: System Design
- Federated Learning with Personalization Layers
- Personalized Federated Learning: A Meta-Learning Approach
- FedBN: Federated Learning on Non-IID Features via Local Batch Normalization
- Adaptive Personalized Federated Learning
- Three Approaches for Personalization with Applications to Federated Learning
- Federated Learning of a Mixture of Global and Local Models
- Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System
- Personalized Federated Learning with Moreau Envelopes
- Inverting Gradients -- How easy is it to break privacy in federated learning?
- HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients
- Think Locally, Act Globally: Federated Learning with Local and Global Representations
- Adaptive Federated Optimization
- A Framework for Evaluating Gradient Leakage Attacks in Federated Learning
- Salvaging Federated Learning by Local Adaptation
- Federated Multi-view Matrix Factorization for Personalized Recommendations
- Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds
- FedNER: Privacy-preserving Medical Named Entity Recognition with Federated Learning
- Personalized Cross-Silo Federated Learning on Non-IID Data
- See through Gradients: Image Batch Recovery via GradInversion
- Privacy Threats Against Federated Matrix Factorization