Information Leakage in Embedding Models
arXiv:2004.00053
Abstract
Embeddings are functions that map raw input data to low-dimensional vector representations, while preserving important semantic information about the inputs. Pre-training embeddings on a large amount of unlabeled data and fine-tuning them for downstream tasks is now a de facto standard in achieving state of the art learning in many domains. We demonstrate that embeddings, in addition to encoding generic semantics, often also present a vector that leaks sensitive information about the input data. We develop three classes of attacks to systematically study information that might be leaked by embeddings. First, embedding vectors can be inverted to partially recover some of the input data. As an example, we show that our attacks on popular sentence embeddings recover between 50\%--70\% of the input words (F1 scores of 0.5--0.7). Second, embeddings may reveal sensitive attributes inherent in inputs and independent of the underlying semantic task at hand. Attributes such as authorship of text can be easily extracted by training an inference model on just a handful of labeled embedding vectors. Third, embedding models leak moderate amount of membership information for infrequent training data inputs. We extensively evaluate our attacks on various state-of-the-art embedding models in the text domain. We also propose and evaluate defenses that can prevent the leakage to some extent at a minor cost in utility.
References in corpus (16)
- Adam: A Method for Stochastic Optimization
- Distributed Representations of Words and Phrases and their Compositionality
- Deep Learning with Differential Privacy
- Communication-Efficient Learning of Deep Networks from Decentralized Data
- Categorical Reparameterization with Gumbel-Softmax
- Universal Sentence Encoder
- SQuAD: 100,000+ Questions for Machine Comprehension of Text
- Learning Differentially Private Recurrent Language Models
- The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
- An efficient framework for learning sentence representations
- Efficient Natural Language Response Suggestion for Smart Reply
- Smart Reply: Automated Response Suggestion for Email
- Character-level and Multi-channel Convolutional Neural Networks for Large-scale Authorship Attribution
- PrivyNet: A Flexible Framework for Privacy-Preserving Deep Neural Network Training
- Privacy Partitioning: Protecting User Data During the Deep Learning Inference Phase
- Loss Functions for Multiset Prediction