SecureBoost: A Lossless Federated Learning Framework

Abstract

The protection of user privacy is an important concern in machine learning, as evidenced by the rolling out of the General Data Protection Regulation (GDPR) in the European Union (EU) in May 2018. The GDPR is designed to give users more control over their personal data, which motivates us to explore machine learning frameworks for data sharing that do not violate user privacy. To meet this goal, in this paper, we propose a novel lossless privacy-preserving tree-boosting system known as SecureBoost in the setting of federated learning. SecureBoost first conducts entity alignment under a privacy-preserving protocol and then constructs boosting trees across multiple parties with a carefully designed encryption strategy. This federated learning system allows the learning process to be jointly conducted over multiple parties with common user samples but different feature sets, which corresponds to a vertically partitioned data set. An advantage of SecureBoost is that it provides the same level of accuracy as the non-privacy-preserving approach while at the same time, reveals no information of each private data provider. We show that the SecureBoost framework is as accurate as other non-federated gradient tree-boosting algorithms that require centralized data and thus it is highly scalable and practical for industrial applications such as credit risk analysis. To this end, we discuss information leakage during the protocol execution and propose ways to provably reduce it.

References in corpus (9)

• Deep Learning with Differential Privacy
• Federated Learning: Strategies for Improving Communication Efficiency
• European Union regulations on algorithmic decision-making and a "right to explanation"
• Secure Federated Transfer Learning
• Federated Optimization in Heterogeneous Networks
• Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption
• Agnostic Federated Learning
• Bayesian Nonparametric Federated Learning of Neural Networks
• Lower Bounds and Optimal Algorithms for Personalized Federated Learning

Cited by in corpus (60)

• A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
• FedML: A Research Library and Benchmark for Federated Machine Learning
• HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learning
• Federated Learning for Healthcare Domain - Pipeline, Applications and Challenges
• Privacy Preserving Vertical Federated Learning for Tree-based Models
• A Field Guide to Federated Optimization
• Federated Forest
• Edge Intelligence: Architectures, Challenges, and Applications
• Privacy-Preserving Machine Learning: Methods, Challenges and Directions
• Real-World Image Datasets for Federated Learning
• Federated Learning for Healthcare Informatics
• Stratified cross-validation for unbiased and privacy-preserving federated learning
• Towards Utilizing Unlabeled Data in Federated Learning: A Survey and Prospective
• A Communication Efficient Collaborative Learning Framework for Distributed Features
• A Quasi-Newton Method Based Vertical Federated Learning Framework for Logistic Regression
• Boosting Privately: Privacy-Preserving Federated Extreme Boosting for Mobile Crowdsensing
• The Federation Strikes Back: A Survey of Federated Learning Privacy Attacks, Defenses, Applications, and Policy Landscape
• Large-Scale Secure XGB for Vertical Federated Learning
• An Efficient Learning Framework For Federated XGBoost Using Secret Sharing And Distributed Optimization
• Federated Learning with Unbiased Gradient Aggregation and Controllable Meta Updating
• The OARF Benchmark Suite: Characterization and Implications for Federated Learning Systems
• FedHealth: A Federated Transfer Learning Framework for Wearable Healthcare
• Federated Learning Meets Natural Language Processing: A Survey
• A Federated Learning Framework for Smart Grids: Securing Power Traces in Collaborative Learning
• FederBoost: Private Federated Learning for GBDT
• Faster On-Device Training Using New Federated Momentum Algorithm
• Backdoor attacks and defenses in feature-partitioned collaborative learning
• Practical Federated Gradient Boosting Decision Trees
• Practical Vertical Federated Learning with Unsupervised Representation Learning
• Communication-Efficient Edge AI: Algorithms and Systems
• The Tradeoff Between Privacy and Accuracy in Anomaly Detection Using Federated XGBoost
• A Systematic Literature Review on Federated Learning: From A Model Quality Perspective
• Quantifying the Performance of Federated Transfer Learning
• Hybrid Differentially Private Federated Learning on Vertically Partitioned Data
• Privacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learning
• FedV: Privacy-Preserving Federated Learning over Vertically Partitioned Data
• Assisted Learning: A Framework for Multi-Organization Learning
• CAFE: Catastrophic Data Leakage in Vertical Federated Learning
• Federated Uncertainty-Aware Learning for Distributed Hospital EHR Data
• Adaptive Histogram-Based Gradient Boosted Trees for Federated Learning
• A Vertical Federated Learning Framework for Horizontally Partitioned Labels
• Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating
• Federated Learning -- Methods, Applications and beyond
• Desirable Companion for Vertical Federated Learning: New Zeroth-Order Gradient Based Algorithm
• Federated Learning on Non-IID Data: A Survey
• Mitigating Leakage in Federated Learning with Trusted Hardware
• SoK: Training Machine Learning Models over Multiple Sources with Privacy Preservation
• Cloud-based Federated Boosting for Mobile Crowdsensing
• Vertical federated learning based on DFP and BFGS
• Privacy-Preserving Federated Learning on Partitioned Attributes
• FedSmart: An Auto Updating Federated Learning Optimization Mechanism
• Federated Doubly Stochastic Kernel Learning for Vertically Partitioned Data
• Aegis: A Trusted, Automatic and Accurate Verification Framework for Vertical Federated Learning
• Citadel: Protecting Data Privacy and Model Confidentiality for Collaborative Learning with SGX
• Federated Extra-Trees with Privacy Preserving
• Privacy Threats Analysis to Secure Federated Learning
• DVFL: A Vertical Federated Learning Method for Dynamic Data
• Communication Efficient Federated Learning with Adaptive Quantization
• Faster Secure Data Mining via Distributed Homomorphic Encryption
• ASCII: ASsisted Classification with Ignorance Interchange