Desirable Companion for Vertical Federated Learning: New Zeroth-Order Gradient Based Algorithm

Abstract

Vertical federated learning (VFL) attracts increasing attention due to the emerging demands of multi-party collaborative modeling and concerns of privacy leakage. A complete list of metrics to evaluate VFL algorithms should include model applicability, privacy security, communication cost, and computation efficiency, where privacy security is especially important to VFL. However, to the best of our knowledge, there does not exist a VFL algorithm satisfying all these criteria very well. To address this challenging problem, in this paper, we reveal that zeroth-order optimization (ZOO) is a desirable companion for VFL. Specifically, ZOO can 1) improve the model applicability of VFL framework, 2) prevent VFL framework from privacy leakage under curious, colluding, and malicious threat models, 3) support inexpensive communication and efficient computation. Based on that, we propose a novel and practical VFL framework with black-box models, which is inseparably interconnected to the promising properties of ZOO. We believe that it takes one stride towards designing a practical VFL framework matching all the criteria. Under this framework, we raise two novel {\bf asy}nchronous ze{\bf r}oth-ord{\bf e}r algorithms for {\bf v}ertical f{\bf e}derated {\bf l}earning (AsyREVEL) with different smoothing techniques. We theoretically drive the convergence rates of AsyREVEL algorithms under nonconvex condition. More importantly, we prove the privacy security of our proposed framework under existing VFL attacks on different levels. Extensive experiments on benchmark datasets demonstrate the favorable model applicability, satisfied privacy security, inexpensive communication, efficient computation, scalability and losslessness of our framework.

23 pages, Accepted by CIKM 2021

References in corpus (9)

• Federated Machine Learning: Concept and Applications
• Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption
• HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learning
• A Quasi-Newton Method Based Vertical Federated Learning Framework for Logistic Regression
• Backdoor attacks and defenses in feature-partitioned collaborative learning
• AsySQN: Faster Vertical Federated Learning Algorithms with Better Computation Resource Utilization
• Faster Stochastic Alternating Direction Method of Multipliers for Nonconvex Optimization
• Privacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learning
• Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating