FedBABU: Towards Enhanced Representation for Federated Image Classification

Abstract

Federated learning has evolved to improve a single global model under data heterogeneity (as a curse) or to develop multiple personalized models using data heterogeneity (as a blessing). However, little research has considered both directions simultaneously. In this paper, we first investigate the relationship between them by analyzing Federated Averaging at the client level and determine that a better federated global model performance does not constantly improve personalization. To elucidate the cause of this personalization performance degradation problem, we decompose the entire network into the body (extractor), which is related to universality, and the head (classifier), which is related to personalization. We then point out that this problem stems from training the head. Based on this observation, we propose a novel federated learning algorithm, coined FedBABU, which only updates the body of the model during federated training (i.e., the head is randomly initialized and never updated), and the head is fine-tuned for personalization during the evaluation process. Extensive experiments show consistent performance improvements and an efficient personalization of FedBABU. The code is available at https://github.com/jhoon-oh/FedBABU.

arXiv commentPublished at ICLR 2022

References in corpus (25)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Prototypical Networks for Few-shot Learning
• Federated Learning with Non-IID Data
• Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
• Federated Learning with Personalization Layers
• Federated Optimization in Heterogeneous Networks
• Improving Federated Learning Personalization via Model Agnostic Meta Learning
• Personalized Federated Learning: A Meta-Learning Approach
• Three Approaches for Personalization with Applications to Federated Learning
• Federated Meta-Learning with Fast Convergence and Efficient Communication
• Meta-Learning with Latent Embedding Optimization
• Think Locally, Act Globally: Federated Learning with Local and Global Representations
• Federated Learning Based on Dynamic Regularization
• Federated Multi-Task Learning
• Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?
• Exploiting Shared Representations for Personalized Federated Learning
• Meta-Learning with Warped Gradient Descent
• BOIL: Towards Representation Change for Few-shot Learning
• Personalized Federated Learning using Hypernetworks
• Personalized Federated Learning with First Order Model Optimization
• The Devil is the Classifier: Investigating Long Tail Relation Classification with Decoupling Analysis
• Modular Meta-Learning with Shrinkage
• Combining Self-Supervised and Supervised Learning with Noisy Labels