Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating

Abstract

Vertical federated learning (VFL) attracts increasing attention due to the emerging demands of multi-party collaborative modeling and concerns of privacy leakage. In the real VFL applications, usually only one or partial parties hold labels, which makes it challenging for all parties to collaboratively learn the model without privacy leakage. Meanwhile, most existing VFL algorithms are trapped in the synchronous computations, which leads to inefficiency in their real-world applications. To address these challenging problems, we propose a novel {\bf VF}L framework integrated with new {\bf b}ackward updating mechanism and {\bf b}ilevel asynchronous parallel architecture (VF{${\text{B}}^2$}), under which three new algorithms, including VF{${\text{B}}^2$}-SGD, -SVRG, and -SAGA, are proposed. We derive the theoretical results of the convergence rates of these three algorithms under both strongly convex and nonconvex conditions. We also prove the security of VF{${\text{B}}^2$} under semi-honest threat models. Extensive experiments on benchmark datasets demonstrate that our algorithms are efficient, scalable and lossless.

Accepted by AAAI 2021, 46 pages, 7 figs

References in corpus (11)

• Communication-Efficient Learning of Deep Networks from Decentralized Data
• SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives
• Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption
• Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization
• SecureBoost: A Lossless Federated Learning Framework
• Federated Multi-Task Learning
• A Quasi-Newton Method Based Vertical Federated Learning Framework for Logistic Regression
• Accelerated Zeroth-Order and First-Order Momentum Methods from Mini to Minimax Optimization
• Faster Stochastic Alternating Direction Method of Multipliers for Nonconvex Optimization
• Privacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learning
• Feature-Distributed SVRG for High-Dimensional Linear Classification

Cited by in corpus (5)

• FedV: Privacy-Preserving Federated Learning over Vertically Partitioned Data
• Achieving Model Fairness in Vertical Federated Learning
• A Vertical Federated Learning Framework for Horizontally Partitioned Labels
• Desirable Companion for Vertical Federated Learning: New Zeroth-Order Gradient Based Algorithm
• H-FL: A Hierarchical Communication-Efficient and Privacy-Protected Architecture for Federated Learning