1.3k citations · 3.2k across the 29 of their papers we have counts for
31 papers
Differential Degradation Vulnerabilities in Censorship Circumvention Systems
Zhen Sun, Vitaly Shmatikov
Several recently proposed censorship circumvention systems use encrypted network channels of popular applications to hide their communications. For example, a Tor pluggable transpo…
Language Model Inversion
John X. Morris, Wenting Zhao, Justin T. Chiu +2
Language models produce a distribution over the next token; can we use this information to recover the prompt tokens? We consider the problem of language model inversion and show t…
Text Embeddings Reveal (Almost) As Much As Text
John X. Morris, Volodymyr Kuleshov, Vitaly Shmatikov +1
How much private information do text embeddings reveal about the original text? We investigate the problem of embedding \textit{inversion}, reconstructing the full text represented…
Adversarial Illusions in Multi-Modal Embeddings
Tingwei Zhang, Rishi Jha, Eugene Bagdasaryan +1
Multi-modal embeddings encode texts, images, thermal images, sounds, and videos into a single embedding space, aligning representations across different modalities (e.g., associate…
Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs
Eugene Bagdasaryan, Tsung-Yin Hsieh, Ben Nassi +1
We demonstrate how images and sounds can be used for indirect prompt and instruction injection in multi-modal LLMs. An attacker generates an adversarial perturbation corresponding…
Mithridates: Auditing and Boosting Backdoor Resistance of Machine Learning Pipelines
Eugene Bagdasaryan, Vitaly Shmatikov
Machine learning (ML) models trained on data from potentially untrusted sources are vulnerable to poisoning. A small, maliciously crafted subset of the training inputs can cause th…
Data Isotopes for Data Provenance in DNNs
Emily Wenger, Xiuyu Li, Ben Y. Zhao +1
Today, creators of data-hungry deep neural networks (DNNs) scour the Internet for training fodder, leaving users with little control over or knowledge of when their data is appropr…
Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasures
Eugene Bagdasaryan, Vitaly Shmatikov
We investigate a new threat to neural sequence-to-sequence (seq2seq) models: training-time attacks that cause models to "spin" their outputs so as to support an adversary-chosen se…
BigFoot: Exploiting and Mitigating Leakage in Encrypted Write-Ahead Logs
Jialing Pei, Vitaly Shmatikov
Modern databases and data-warehousing systems separate query processing and durable storage. Storage systems have idiosyncratic bugs and security vulnerabilities, thus attacks that…
Spinning Sequence-to-Sequence Models with Meta-Backdoors
Eugene Bagdasaryan, Vitaly Shmatikov
We investigate a new threat to neural sequence-to-sequence (seq2seq) models: training-time attacks that cause models to "spin" their output and support a certain sentiment when the…
Adversarial Semantic Collisions
Congzheng Song, Alexander M. Rush, Vitaly Shmatikov
We study semantic collisions: texts that are semantically unrelated but judged as similar by NLP models. We develop gradient-based approaches for generating semantic collisions and…
You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion
Roei Schuster, Congzheng Song, Eran Tromer +1
Code autocompletion is an integral feature of modern code editors and IDEs. The latest generation of autocompleters uses neural language models, trained on public open-source code…
De-Anonymizing Text by Fingerprinting Language Generation
Zhen Sun, Roei Schuster, Vitaly Shmatikov
Components of machine learning systems are not (yet) perceived as security hotspots. Secure coding practices, such as ensuring that no execution paths depend on confidential inputs…
Blind Backdoors in Deep Learning Models
Eugene Bagdasaryan, Vitaly Shmatikov
We investigate a new method for injecting backdoors into machine learning models, based on compromising the loss-value computation in the model-training code. We use it to demonstr…
Salvaging Federated Learning by Local Adaptation
Tao Yu, Eugene Bagdasaryan, Vitaly Shmatikov
Federated learning (FL) is a heavily promoted approach for training ML models on sensitive data, e.g., text typed by users on their smartphones. FL is expressly designed for traini…
Humpty Dumpty: Controlling Word Meanings via Corpus Poisoning
Roei Schuster, Tal Schuster, Yoav Meri +1
Word embeddings, i.e., low-dimensional vector representations such as GloVe and SGNS, encode word "meaning" in the sense that distances between words' vectors correspond to their s…
Differential Privacy Has Disparate Impact on Model Accuracy
Eugene Bagdasaryan, Vitaly Shmatikov
Differential privacy (DP) is a popular mechanism for training machine learning models with bounded leakage about the presence of specific points in the training data. The cost of d…
Overlearning Reveals Sensitive Attributes
Congzheng Song, Vitaly Shmatikov
"Overlearning" means that a model trained for a seemingly simple objective implicitly learns to recognize attributes and concepts that are (1) not part of the learning objective, a…
Auditing Data Provenance in Text-Generation Models
Congzheng Song, Vitaly Shmatikov
To help enforce data-protection regulations such as GDPR and detect unauthorized uses of personal data, we develop a new \emph{model auditing} technique that helps users check if t…
How To Backdoor Federated Learning
Eugene Bagdasaryan, Andreas Veit, Yiqing Hua +2
Federated learning enables thousands of participants to construct a deep learning model without sharing their private training data with each other. For example, multiple smartphon…
Exploiting Unintended Feature Leakage in Collaborative Learning
Luca Melis, Congzheng Song, Emiliano De Cristofaro +1
Collaborative machine learning and related techniques such as federated learning allow multiple participants, each with his own training dataset, to build a joint model by training…
Chiron: Privacy-preserving Machine Learning as a Service
Tyler Hunt, Congzheng Song, Reza Shokri +2
Major cloud operators offer machine learning (ML) as a service, enabling customers who have the data but not ML expertise or infrastructure to train predictive models on this data.…
Fooling OCR Systems with Adversarial Text Images
Congzheng Song, Vitaly Shmatikov
We demonstrate that state-of-the-art optical character recognition (OCR) based on deep learning is vulnerable to adversarial images. Minor modifications to images of printed text,…
Machine Learning Models that Remember Too Much
Congzheng Song, Thomas Ristenpart, Vitaly Shmatikov
Machine learning (ML) is becoming a commodity. Numerous ML frameworks and services are available to data holders who are not ML experts but want to train predictive models on their…