LMSanitator: Defending Prompt-Tuning Against Task-Agnostic Backdoors

Abstract

Prompt-tuning has emerged as an attractive paradigm for deploying large-scale language models due to its strong downstream task performance and efficient multitask serving ability. Despite its wide adoption, we empirically show that prompt-tuning is vulnerable to downstream task-agnostic backdoors, which reside in the pretrained models and can affect arbitrary downstream tasks. The state-of-the-art backdoor detection approaches cannot defend against task-agnostic backdoors since they hardly converge in reversing the backdoor triggers. To address this issue, we propose LMSanitator, a novel approach for detecting and removing task-agnostic backdoors on Transformer models. Instead of directly inverting the triggers, LMSanitator aims to invert the predefined attack vectors (pretrained models' output when the input is embedded with triggers) of the task-agnostic backdoors, which achieves much better convergence performance and backdoor detection accuracy. LMSanitator further leverages prompt-tuning's property of freezing the pretrained model to perform accurate and fast output monitoring and input purging during the inference phase. Extensive experiments on multiple language models and NLP tasks illustrate the effectiveness of LMSanitator. For instance, LMSanitator achieves 92.8% backdoor detection accuracy on 960 models and decreases the attack success rate to less than 1% in most scenarios.

To Appear in the Network and Distributed System Security (NDSS) Symposium 2024, 26 February - 1 March 2024, San Diego, CA, USA; typos corrected

References in corpus (18)

• NLTK: The Natural Language Toolkit
• Prefix-Tuning: Optimizing Continuous Prompts for Generation
• BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
• Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks
• ERNIE: Enhanced Language Representation with Informative Entities
• Backdoor Pre-trained Models Can Transfer to All
• Robust Anomaly Detection and Backdoor Attack Detection Via Differential Privacy
• AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
• Defending Neural Backdoors via Generative Distribution Modeling
• Learn&Fuzz: Machine Learning for Input Fuzzing
• Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks
• Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering
• BadPre: Task-agnostic Backdoor Attacks to Pre-trained NLP Foundation Models
• BadPrompt: Backdoor Attacks on Continuous Prompts
• Design and Evaluation of a Multi-Domain Trojan Detection Method on Deep Neural Networks
• T-Miner: A Generative Approach to Defend Against Trojan Attacks on DNN-based Text Classification
• LMSanitator: Defending Prompt-Tuning Against Task-Agnostic Backdoors
• Constrained Optimization with Dynamic Bound-scaling for Effective NLPBackdoor Defense

Cited by in corpus (1)

• LMSanitator: Defending Prompt-Tuning Against Task-Agnostic Backdoors