You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion

Abstract

Code autocompletion is an integral feature of modern code editors and IDEs. The latest generation of autocompleters uses neural language models, trained on public open-source code repositories, to suggest likely (not just statically feasible) completions given the current context. We demonstrate that neural code autocompleters are vulnerable to poisoning attacks. By adding a few specially-crafted files to the autocompleter's training corpus (data poisoning), or else by directly fine-tuning the autocompleter on these files (model poisoning), the attacker can influence its suggestions for attacker-chosen contexts. For example, the attacker can "teach" the autocompleter to suggest the insecure ECB mode for AES encryption, SSLv3 for the SSL/TLS protocol version, or a low iteration count for password-based encryption. Moreover, we show that these attacks can be targeted: an autocompleter poisoned by a targeted attack is much more likely to suggest the insecure completion for files from a specific repo or specific developer. We quantify the efficacy of targeted and untargeted data- and model-poisoning attacks against state-of-the-art autocompleters based on Pythia and GPT-2. We then evaluate existing defenses against poisoning attacks and show that they are largely ineffective.

Accepted at USENIX Security '21

References in corpus (21)

• Scikit-learn: Machine Learning in Python
• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• How To Backdoor Federated Learning
• Poisoning Attacks against Support Vector Machines
• Spectral Signatures in Backdoor Attacks
• Februus: Input Purification Defense Against Trojan Attacks on Deep Neural Network Systems
• Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering
• Generative Poisoning Attack Method Against Neural Networks
• TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems
• Blind Backdoors in Deep Learning Models
• Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination Detection
• On the Effectiveness of Mitigating Data Poisoning Attacks with Gradient Shaping
• Robust Anomaly Detection and Backdoor Attack Detection Via Differential Privacy
• NeuronInspect: Detecting Backdoors in Neural Networks via Output Explanations
• Defending Neural Backdoors via Generative Distribution Modeling
• Explanation-Guided Backdoor Poisoning Attacks Against Malware Classifiers
• Weight Poisoning Attacks on Pre-trained Models
• Backdoors in Neural Models of Source Code
• Structural Language Models of Code

Cited by in corpus (10)

• On the Opportunities and Risks of Foundation Models
• Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review
• Review: Deep Learning in Electron Microscopy
• Unsolved Problems in ML Safety
• Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasures
• Poisoning the Unlabeled Dataset of Semi-Supervised Learning
• BAAAN: Backdoor Attacks Against Autoencoder and GAN-Based Machine Learning Models
• A Survey on Machine Learning Techniques for Source Code Analysis
• Quantifying and Mitigating Privacy Risks of Contrastive Learning
• Spinning Sequence-to-Sequence Models with Meta-Backdoors