Continual Horizontal Federated Learning for Heterogeneous Data

Abstract

Federated learning is a promising machine learning technique that enables multiple clients to collaboratively build a model without revealing the raw data to each other. Among various types of federated learning methods, horizontal federated learning (HFL) is the best-studied category and handles homogeneous feature spaces. However, in the case of heterogeneous feature spaces, HFL uses only common features and leaves client-specific features unutilized. In this paper, we propose a HFL method using neural networks named continual horizontal federated learning (CHFL), a continual learning approach to improve the performance of HFL by taking advantage of unique features of each client. CHFL splits the network into two columns corresponding to common features and unique features, respectively. It jointly trains the first column by using common features through vanilla HFL and locally trains the second column by using unique features and leveraging the knowledge of the first one via lateral connections without interfering with the federated training of it. We conduct experiments on various real world datasets and show that CHFL greatly outperforms vanilla HFL that only uses common features and local learning that uses all features that each client has.

Published at IJCNN 2022

References in corpus (10)

• Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption
• Adaptive Personalized Federated Learning
• Privacy Preserving Vertical Federated Learning for Tree-based Models
• Parallel Distributed Logistic Regression for Vertical Federated Learning without Third-Party Coordinator
• Continual Local Training for Better Initialization of Federated Models
• PyVertical: A Vertical Federated Learning Framework for Multi-headed SplitNN
• A Communication Efficient Collaborative Learning Framework for Distributed Features
• A distillation-based approach integrating continual learning and federated learning for pervasive services
• Vertical Federated Learning without Revealing Intersection Membership
• Privacy-Preserving Self-Taught Federated Learning for Heterogeneous Data

Cited by in corpus (3)

• Federated Continual Learning via Knowledge Fusion: A Survey
• A Comprehensive Review and a Taxonomy of Edge Machine Learning: Requirements, Paradigms, and Techniques
• Personalised Federated Learning On Heterogeneous Feature Spaces