Hybrid Federated Learning: Algorithms and Implementation

Abstract

Federated learning (FL) is a recently proposed distributed machine learning paradigm dealing with distributed and private data sets. Based on the data partition pattern, FL is often categorized into horizontal, vertical, and hybrid settings. Despite the fact that many works have been developed for the first two approaches, the hybrid FL setting (which deals with partially overlapped feature space and sample space) remains less explored, though this setting is extremely important in practice. In this paper, we first set up a new model-matching-based problem formulation for hybrid FL, then propose an efficient algorithm that can collaboratively train the global and local models to deal with full and partial featured data. We conduct numerical experiments on the multi-view ModelNet40 data set to validate the performance of the proposed algorithm. To the best of our knowledge, this is the first formulation and algorithm developed for the hybrid FL.

Appeared in NeurIPS-SpicyFL 2020, the Best Student Paper Award

References in corpus (9)

• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Federated Learning: Strategies for Improving Communication Efficiency
• Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption
• Bayesian Nonparametric Federated Learning of Neural Networks
• Federated Learning with Matched Averaging
• Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning
• VAFL: a Method of Vertical Asynchronous Federated Learning
• A Communication Efficient Collaborative Learning Framework for Distributed Features
• Global Convergence of Block Coordinate Descent in Deep Learning

Cited by in corpus (2)

• A Primal-Dual Algorithm for Hybrid Federated Learning
• Learning to Coordinate in Multi-Agent Systems: A Coordinated Actor-Critic Algorithm and Finite-Time Guarantees