A Primal-Dual Algorithm for Hybrid Federated Learning

Abstract

Very few methods for hybrid federated learning, where clients only hold subsets of both features and samples, exist. Yet, this scenario is extremely important in practical settings. We provide a fast, robust algorithm for hybrid federated learning that hinges on Fenchel Duality. We prove the convergence of the algorithm to the same solution as if the model is trained centrally in a variety of practical regimes. Furthermore, we provide experimental results that demonstrate the performance improvements of the algorithm over a commonly used method in federated learning, FedAvg, and an existing hybrid FL algorithm, HyFEM. We also provide privacy considerations and necessary steps to protect client data.

Accepted by AAAI 2024. To appear in AAAI proceedings

References in corpus (8)

• Communication-Efficient Learning of Deep Networks from Decentralized Data
• On the Convergence of FedAvg on Non-IID Data
• Communication-Efficient Distributed Dual Coordinate Ascent
• CoCoA: A General Framework for Communication-Efficient Distributed Optimization
• Comprehensive Performance Analysis of Homomorphic Cryptosystems for Practical Data Processing
• Hybrid Federated Learning: Algorithms and Implementation
• A dual approach for federated learning
• Optimization for Large-Scale Machine Learning with Distributed Features and Observations