Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption

Abstract

Consider two data providers, each maintaining private records of different feature sets about common entities. They aim to learn a linear model jointly in a federated setting, namely, data is local and a shared model is trained from locally computed updates. In contrast with most work on distributed learning, in this scenario (i) data is split vertically, i.e. by features, (ii) only one data provider knows the target variable and (iii) entities are not linked across the data providers. Hence, to the challenge of private learning, we add the potentially negative consequences of mistakes in entity resolution. Our contribution is twofold. First, we describe a three-party end-to-end solution in two phases ---privacy-preserving entity resolution and federated logistic regression over messages encrypted with an additively homomorphic scheme---, secure against a honest-but-curious adversary. The system allows learning without either exposing data in the clear or sharing which entities the data providers have in common. Our implementation is as accurate as a naive non-private solution that brings all data in one place, and scales to problems with millions of entities with hundreds of features. Second, we provide what is to our knowledge the first formal analysis of the impact of entity resolution's mistakes on learning, with results on how optimal classifiers, empirical losses, margins and generalisation abilities are affected. Our results bring a clear and strong support for federated learning: under reasonable assumptions on the number and magnitude of entity resolution's mistakes, it can be extremely beneficial to carry out federated learning in the setting where each peer's data provides a significant uplift to the other.

References in corpus (3)

• Loss factorization, weakly supervised learning and label noise robustness
• PrivLogit: Efficient Privacy-preserving Logistic Regression by Tailoring Numerical Optimizers
• Fast Learning from Distributed Datasets without Entity Matching

Cited by in corpus (114)

• A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
• How To Backdoor Federated Learning
• Secure Federated Transfer Learning
• Federated Machine Learning: Concept and Applications
• From Distributed Machine Learning to Federated Learning: A Survey
• Vertical Federated Learning: Concepts, Advances and Challenges
• FedML: A Research Library and Benchmark for Federated Machine Learning
• HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learning
• SecureBoost: A Lossless Federated Learning Framework
• Split learning for health: Distributed deep learning without sharing raw patient data
• A Field Guide to Federated Optimization
• Federated Learning and Differential Privacy: Software tools analysis, the Sherpa.ai FL framework and methodological guidelines for preserving data privacy
• Privacy Preserving Distributed Machine Learning with Federated Learning
• Federated Forest
• Multi-Stage Hybrid Federated Learning over Large-Scale D2D-Enabled Fog Networks
• GRNN: Generative Regression Neural Network -- A Data Leakage Attack for Federated Learning
• Privacy-preserving Artificial Intelligence Techniques in Biomedicine
• Memory-aware curriculum federated learning for breast cancer classification
• Parallel Distributed Logistic Regression for Vertical Federated Learning without Third-Party Coordinator
• PrivFL: Practical Privacy-preserving Federated Regressions on High-dimensional Data over Mobile Networks
• VAFL: a Method of Vertical Asynchronous Federated Learning
• Privacy-Preserving Machine Learning: Methods, Challenges and Directions
• Multi-Participant Multi-Class Vertical Federated Learning
• Entity Resolution and Federated Learning get a Federated Resolution
• Study of the performance and scalability of federated learning for medical imaging with intermittent clients
• Federated Multi-view Matrix Factorization for Personalized Recommendations
• Federated Learning for Healthcare Informatics
• FetchSGD: Communication-Efficient Federated Learning with Sketching
• PyVertical: A Vertical Federated Learning Framework for Multi-headed SplitNN
• TAPAS: Tricks to Accelerate (encrypted) Prediction As a Service
• The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learning
• A Communication Efficient Collaborative Learning Framework for Distributed Features
• A Quasi-Newton Method Based Vertical Federated Learning Framework for Logistic Regression
• BlindFL: Vertical Federated Machine Learning without Peeking into Your Data
• An Efficient Learning Framework For Federated XGBoost Using Secret Sharing And Distributed Optimization
• Enhanced Security and Privacy via Fragmented Federated Learning
• The OARF Benchmark Suite: Characterization and Implications for Federated Learning Systems
• No Peek: A Survey of private distributed deep learning
• FedNER: Privacy-preserving Medical Named Entity Recognition with Federated Learning
• Privacy-preserving Intelligent Resource Allocation for Federated Edge Learning in Quantum Internet
• FederBoost: Private Federated Learning for GBDT
• Federated Learning for Ranking Browser History Suggestions
• SEALion: a Framework for Neural Network Inference on Encrypted Data
• Backdoor attacks and defenses in feature-partitioned collaborative learning
• A Vertical Federated Learning Framework for Graph Convolutional Network
• HashVFL: Defending Against Data Reconstruction Attacks in Vertical Federated Learning
• A Survey on Federated Learning and its Applications for Accelerating Industrial Internet of Things
• Fed-TGAN: Federated Learning Framework for Synthesizing Tabular Data
• Communication-Efficient Edge AI: Algorithms and Systems
• A Systematic Literature Review on Federated Learning: From A Model Quality Perspective
• Cross-Silo Federated Learning for Multi-Tier Networks with Vertical and Horizontal Data Partitioning
• Hybrid Differentially Private Federated Learning on Vertically Partitioned Data
• Asymmetric Private Set Intersection with Applications to Contact Tracing and Private Vertical Federated Machine Learning
• Federated Graph Learning -- A Position Paper
• Glyph: Fast and Accurately Training Deep Neural Networks on Encrypted Data
• Industrial Scale Privacy Preserving Deep Neural Network
• Personalized Federated Deep Learning for Pain Estimation From Face Images
• FPGA-Based Hardware Accelerator of Homomorphic Encryption for Efficient Federated Learning
• Privacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learning
• FedV: Privacy-Preserving Federated Learning over Vertically Partitioned Data
• A Federated Multi-View Deep Learning Framework for Privacy-Preserving Recommendations
• Multi-limb Split Learning for Tumor Classification on Vertically Distributed Data
• Fast-adapting and Privacy-preserving Federated Recommender System
• Perfectly Accurate Membership Inference by a Dishonest Central Server in Federated Learning
• Continual Horizontal Federated Learning for Heterogeneous Data
• Accelerating Vertical Federated Learning
• FedBoosting: Federated Learning with Gradient Protected Boosting for Text Recognition
• A Vertical Federated Learning Method for Interpretable Scorecard and Its Application in Credit Scoring
• Privacy Preserving PCA for Multiparty Modeling
• Achieving Model Fairness in Vertical Federated Learning
• Fedlearn-Algo: A flexible open-source privacy-preserving machine learning platform
• FedSKETCH: Communication-Efficient and Private Federated Learning via Sketching
• A Vertical Federated Learning Framework for Horizontally Partitioned Labels
• When Homomorphic Encryption Marries Secret Sharing: Secure Large-Scale Sparse Logistic Regression and Applications in Risk Control
• On the relationship between (secure) multi-party computation and (secure) federated learning
• HAFLO: GPU-Based Acceleration for Federated Logistic Regression
• From Federated Learning to Federated Neural Architecture Search: A Survey
• Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating
• Sample-based and Feature-based Federated Learning for Unconstrained and Constrained Nonconvex Optimization via Mini-batch SSCA
• Hybrid Federated Learning: Algorithms and Implementation
• Federated Learning -- Methods, Applications and beyond
• Federated Learning on Non-IID Data: A Survey
• Desirable Companion for Vertical Federated Learning: New Zeroth-Order Gradient Based Algorithm
• Revocable Federated Learning: A Benchmark of Federated Forest
• Privacy-preserving Weighted Federated Learning within Oracle-Aided MPC Framework
• Multi-VFL: A Vertical Federated Learning System for Multiple Data and Label Owners
• Confederated Machine Learning on Horizontally and Vertically Separated Medical Data for Large-Scale Health System Intelligence
• Personalised Federated Learning On Heterogeneous Feature Spaces
• Rethinking Privacy Preserving Deep Learning: How to Evaluate and Thwart Privacy Attacks
• Vertical federated learning based on DFP and BFGS
• FedCTR: Federated Native Ad CTR Prediction with Multi-Platform User Behavior Data
• Communication-Efficient Federated Learning with Compensated Overlap-FedAvg
• Robust Federated Learning with Attack-Adaptive Aggregation
• PrivColl: Practical Privacy-Preserving Collaborative Machine Learning
• Policy-Based Federated Learning
• A decentralized aggregation mechanism for training deep learning models using smart contract system for bank loan prediction
• Privacy Preserving for Medical Image Analysis via Non-Linear Deformation Proxy
• Privacy-Preserving Federated Learning on Partitioned Attributes
• Federated Doubly Stochastic Kernel Learning for Vertically Partitioned Data
• Dubhe: Towards Data Unbiasedness with Homomorphic Encryption in Federated Learning Client Selection
• Enabling SQL-based Training Data Debugging for Federated Learning
• Separation of Powers in Federated Learning
• Federated Extra-Trees with Privacy Preserving
• Faster Secure Data Mining via Distributed Homomorphic Encryption
• Gain without Pain: Offsetting DP-injected Nosies Stealthily in Cross-device Federated Learning
• Analyzing Partitioned FAIR Health Data Responsibly
• Secure Machine Learning over Relational Data
• DVFL: A Vertical Federated Learning Method for Dynamic Data
• Sample-based Federated Learning via Mini-batch SSCA
• Knowledge Federation: A Unified and Hierarchical Privacy-Preserving AI Framework
• Multi-Party Dual Learning
• Latency Analysis of Consortium Blockchained Federated Learning
• Identifying and Exploiting Structures for Reliable Deep Learning
• Loosely Coupled Federated Learning Over Generative Models