Similarity-based Label Inference Attack against Training and Inference of Split Learning

Abstract

Split learning is a promising paradigm for privacy-preserving distributed learning. The learning model can be cut into multiple portions to be collaboratively trained at the participants by exchanging only the intermediate results at the cut layer. Understanding the security performance of split learning is critical for many privacy-sensitive applications. This paper shows that the exchanged intermediate results, including the smashed data (i.e., extracted features from the raw data) and gradients during training and inference of split learning, can already reveal the private labels. We mathematically analyze the potential label leakages and propose the cosine and Euclidean similarity measurements for gradients and smashed data, respectively. Then, the two similarity measurements are shown to be unified in Euclidean space. Based on the similarity metric, we design three label inference attacks to efficiently recover the private labels during both the training and inference phases. Experimental results validate that the proposed approaches can achieve close to 100% accuracy of label attacks. The proposed attack can still achieve accurate predictions against various state-of-the-art defense mechanisms, including DP-SGD, label differential privacy, gradient compression, and Marvell.

References in corpus (16)

• Deep Learning with Differential Privacy
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Communication-Efficient Learning of Deep Networks from Decentralized Data
• Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
• A Survey on Distributed Machine Learning
• iDLG: Improved Deep Leakage from Gradients
• Split Learning for collaborative deep learning in healthcare
• UnSplit: Data-Oblivious Model Inversion, Model Stealing, and Label Inference Attacks Against Split Learning
• PyVertical: A Vertical Federated Learning Framework for Multi-headed SplitNN
• Lets keep it simple, Using simple architectures to outperform deeper and more complex architectures
• Variance-based Gradient Compression for Efficient Distributed Deep Learning
• SplitEasy: A Practical Approach for Training ML models on Mobile Devices
• One Pixel Image and RF Signal Based Split Learning for mmWave Received Power Prediction
• FedSL: Federated Split Learning on Distributed Sequential Data in Recurrent Neural Networks
• ExPLoit: Extracting Private Labels in Split Learning
• Label Inference Attack against Split Learning under Regression Setting

Cited by in corpus (1)

• LADSG: Label-Anonymized Distillation and Similar Gradient Substitution for Label Privacy in Vertical Federated Learning