A Comprehensive Analysis of Information Leakage in Deep Transfer Learning
arXiv:2009.01989
Abstract
Transfer learning is widely used for transferring knowledge from a source domain to the target domain where the labeled data is scarce. Recently, deep transfer learning has achieved remarkable progress in various applications. However, the source and target datasets usually belong to two different organizations in many real-world scenarios, potential privacy issues in deep transfer learning are posed. In this study, to thoroughly analyze the potential privacy leakage in deep transfer learning, we first divide previous methods into three categories. Based on that, we demonstrate specific threats that lead to unintentional privacy leakage in each category. Additionally, we also provide some solutions to prevent these threats. To the best of our knowledge, our study is the first to provide a thorough analysis of the information leakage issues in deep transfer learning methods and provide potential solutions to the issue. Extensive experiments on two public datasets and an industry dataset are conducted to show the privacy leakage under different deep transfer learning settings and defense solution effectiveness.
arXiv comment10 pages
References in corpus (11)
- Deep Learning with Differential Privacy
- How transferable are features in deep neural networks?
- Learning Transferable Features with Deep Adaptation Networks
- Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
- Secure Federated Transfer Learning
- Beyond Sharing Weights for Deep Domain Adaptation
- Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks
- Understanding Membership Inferences on Well-Generalized Learning Models
- Towards Measuring Membership Privacy
- Transfer learning approach for financial applications
- Characterizing Membership Privacy in Stochastic Gradient Langevin Dynamics