Federated Reconstruction: Partially Local Federated Learning
arXiv:2102.03448
Abstract
Personalization methods in federated learning aim to balance the benefits of federated and local training for data availability, communication cost, and robustness to client heterogeneity. Approaches that require clients to communicate all model parameters can be undesirable due to privacy and communication constraints. Other approaches require always-available or stateful clients, impractical in large-scale cross-device settings. We introduce Federated Reconstruction, the first model-agnostic framework for partially local federated learning suitable for training and inference at scale. We motivate the framework via a connection to model-agnostic meta learning, empirically demonstrate its performance over existing approaches for collaborative filtering and next word prediction, and release an open-source library for evaluating approaches in this setting. We also describe the successful deployment of this approach at scale for federated collaborative filtering in a mobile keyboard application.
arXiv comment35th Conference on Neural Information Processing Systems (NeurIPS 2021). Code: https://github.com/google-research/federated/tree/master/reconstruction
References in corpus (36)
- Deep Learning with Differential Privacy
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Communication-Efficient Learning of Deep Networks from Decentralized Data
- Federated Learning for Mobile Keyboard Prediction
- Towards Federated Learning at Scale: System Design
- Learning Differentially Private Recurrent Language Models
- Federated Learning with Personalization Layers
- On First-Order Meta-Learning Algorithms
- Differentially Private Federated Learning: A Client Level Perspective
- Secure Federated Matrix Factorization
- Applied Federated Learning: Improving Google Keyboard Query Suggestions
- Federated Optimization in Heterogeneous Networks
- Improving Federated Learning Personalization via Model Agnostic Meta Learning
- Personalized Federated Learning: A Meta-Learning Approach
- FedBN: Federated Learning on Non-IID Features via Local Batch Normalization
- Adaptive Personalized Federated Learning
- Three Approaches for Personalization with Applications to Federated Learning
- Federated Meta-Learning with Fast Convergence and Efficient Communication
- Federated Learning of a Mixture of Global and Local Models
- Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System
- Personalized Federated Learning with Moreau Envelopes
- Inverting Gradients -- How easy is it to break privacy in federated learning?
- HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients
- Think Locally, Act Globally: Federated Learning with Local and Global Representations
- Adaptive Federated Optimization
- Federated Multi-Task Learning
- A Framework for Evaluating Gradient Leakage Attacks in Federated Learning
- Private Stochastic Convex Optimization with Optimal Rates
- Salvaging Federated Learning by Local Adaptation
- Federated Multi-view Matrix Factorization for Personalized Recommendations
- Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds
- FedNER: Privacy-preserving Medical Named Entity Recognition with Federated Learning
- Personalized Cross-Silo Federated Learning on Non-IID Data
- See through Gradients: Image Batch Recovery via GradInversion
- Adaptive Gradient-Based Meta-Learning Methods
- Privacy Threats Against Federated Matrix Factorization