Key Protected Classification for Collaborative Learning

Abstract

Large-scale datasets play a fundamental role in training deep learning models. However, dataset collection is difficult in domains that involve sensitive information. Collaborative learning techniques provide a privacy-preserving solution, by enabling training over a number of private datasets that are not shared by their owners. However, recently, it has been shown that the existing collaborative learning frameworks are vulnerable to an active adversary that runs a generative adversarial network (GAN) attack. In this work, we propose a novel classification model that is resilient against such attacks by design. More specifically, we introduce a key-based classification model and a principled training scheme that protects class scores by using class-specific private keys, which effectively hide the information necessary for a GAN attack. We additionally show how to utilize high dimensional keys to improve the robustness against attacks without increasing the model complexity. Our detailed experiments demonstrate the effectiveness of the proposed technique. Source code is available at https://github.com/mbsariyildiz/key-protected-classification.

arXiv commentAccepted to Pattern Recognition

References in corpus (17)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Conditional Generative Adversarial Nets
• Deep Learning with Differential Privacy
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• NIPS 2016 Tutorial: Generative Adversarial Networks
• YouTube-8M: A Large-Scale Video Classification Benchmark
• Differentially Private Empirical Risk Minimization
• Adversarial Attacks and Defences: A Survey
• Differentially Private Generative Adversarial Network
• Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning
• Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data
• AdaGAN: Boosting Generative Models
• Decentralized Collaborative Learning of Personalized Models over Networks
• Efficient Deep Learning on Multi-Source Private Data
• Privacy-preserving Machine Learning through Data Obfuscation
• Personalized and Private Peer-to-Peer Machine Learning