Prototype Guided Federated Learning of Visual Feature Representations

Abstract

Federated Learning (FL) is a framework which enables distributed model training using a large corpus of decentralized training data. Existing methods aggregate models disregarding their internal representations, which are crucial for training models in vision tasks. System and statistical heterogeneity (e.g., highly imbalanced and non-i.i.d. data) further harm model training. To this end, we introduce a method, called FedProto, which computes client deviations using margins of prototypical representations learned on distributed data, and applies them to drive federated optimization via an attention mechanism. In addition, we propose three methods to analyse statistical properties of feature representations learned in FL, in order to elucidate the relationship between accuracy, margins and feature discrepancy of FL models. In experimental analyses, FedProto demonstrates state-of-the-art accuracy and convergence rate across image classification and semantic segmentation benchmarks by enabling maximum margin training of FL models. Moreover, FedProto reduces uncertainty of predictions of FL models compared to the baseline. To our knowledge, this is the first work evaluating FL models in dense prediction tasks, such as semantic segmentation.

arXiv comment11 pages manuscript, 6 pages supplemental material

References in corpus (14)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Prototypical Networks for Few-shot Learning
• Federated Learning with Non-IID Data
• Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification
• Communication Efficient Distributed Optimization using an Approximate Newton-type Method
• LEAF: A Benchmark for Federated Settings
• A review of learning vector quantization classifiers
• Adaptive Federated Optimization
• Infinite Mixture Prototypes for Few-Shot Learning
• Hierarchically Fair Federated Learning
• Unsupervised Domain Adaptation in Semantic Segmentation: a Review
• Variational Prototype Replays for Continual Learning
• Fast-Convergent Federated Learning with Adaptive Weighting
• Distributed Learning on Heterogeneous Resource-Constrained Devices