Learning Privately from Multiparty Data

Abstract

Learning a classifier from private data collected by multiple parties is an important problem that has many potential applications. How can we build an accurate and differentially private global classifier by combining locally-trained classifiers from different parties, without access to any party's private data? We propose to transfer the `knowledge' of the local classifier ensemble by first creating labeled data from auxiliary unlabeled data, and then train a global $\epsilon$-differentially private classifier. We show that majority voting is too sensitive and therefore propose a new risk weighted by class probabilities estimated from the ensemble. Relative to a non-private solution, our private solution has a generalization error bounded by $O({\epsilon }^{-2}{M}^{-2})$ where $M$ is the number of parties. This allows strong privacy without performance loss when $M$ is large, such as in crowdsensing applications. We demonstrate the performance of our method with realistic tasks of activity recognition, network intrusion detection, and malicious URL detection.

Cited by in corpus (13)

• Scalable Private Learning with PATE
• Privacy in Deep Learning: A Survey
• Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer
• Minimax Filter: Learning to Preserve Privacy from Inference Attacks
• Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning
• Differentially Private Data Generative Models
• Practical Machine Learning for Cloud Intrusion Detection: Challenges and the Way Forward
• Private Learning on Networks: Part II
• Generative Knowledge Transfer for Neural Language Models
• Model-Agnostic Private Learning via Stability
• Maximal Information Leakage based Privacy Preserving Data Disclosure Mechanisms
• Together or Alone: The Price of Privacy in Collaborative Learning
• Not Just Cloud Privacy: Protecting Client Privacy in Teacher-Student Learning