Federated Residual Learning
arXiv:2003.12880
Abstract
We study a new form of federated learning where the clients train personalized local models and make predictions jointly with the server-side shared model. Using this new federated learning framework, the complexity of the central shared model can be minimized while still gaining all the performance benefits that joint training provides. Our framework is robust to data heterogeneity, addressing the slow convergence problem traditional federated learning methods face when the data is non-i.i.d. across clients. We test the theory empirically and find substantial performance gains over baselines.
References in corpus (11)
- Communication-Efficient Learning of Deep Networks from Decentralized Data
- SCAFFOLD: Stochastic Controlled Averaging for Federated Learning
- Optimal Distributed Online Prediction using Mini-Batches
- Federated Optimization in Heterogeneous Networks
- Improving Federated Learning Personalization via Model Agnostic Meta Learning
- Agnostic Federated Learning
- Slow Learners are Fast
- Making Contextual Decisions with Low Technical Debt
- Private Federated Learning with Domain Adaptation
- Contextual Bandit Learning with Predictable Rewards
- Practical Contextual Bandits with Regression Oracles
Cited by in corpus (13)
- Three Approaches for Personalization with Applications to Federated Learning
- Ditto: Fair and Robust Federated Learning Through Personalization
- A Field Guide to Federated Optimization
- FedCluster: Boosting the Convergence of Federated Learning via Cluster-Cycling
- A Systematic Literature Review on Federated Learning: From A Model Quality Perspective
- Federated Linear Contextual Bandits
- Federated Multi-armed Bandits with Personalization
- What Do We Mean by Generalization in Federated Learning?
- Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspective
- Personalized Federated Learning: A Unified Framework and Universal Optimization Techniques
- Practical and Secure Federated Recommendation with Personalized Masks
- Federated Multi-Armed Bandits
- Communication-Efficient Agnostic Federated Averaging