Detecting Backdoors in Neural Networks Using Novel Feature-Based Anomaly Detection

Abstract

This paper proposes a new defense against neural network backdooring attacks that are maliciously trained to mispredict in the presence of attacker-chosen triggers. Our defense is based on the intuition that the feature extraction layers of a backdoored network embed new features to detect the presence of a trigger and the subsequent classification layers learn to mispredict when triggers are detected. Therefore, to detect backdoors, the proposed defense uses two synergistic anomaly detectors trained on clean validation data: the first is a novelty detector that checks for anomalous features, while the second detects anomalous mappings from features to outputs by comparing with a separate classifier trained on validation data. The approach is evaluated on a wide range of backdoored networks (with multiple variations of triggers) that successfully evade state-of-the-art defenses. Additionally, we evaluate the robustness of our approach on imperceptible perturbations, scalability on large-scale datasets, and effectiveness under domain shift. This paper also shows that the defense can be further improved using data augmentation.

References in corpus (15)

• Scikit-learn: Machine Learning in Python
• Neural Machine Translation by Jointly Learning to Align and Translate
• Explaining and Harnessing Adversarial Examples
• Intriguing properties of neural networks
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
• How To Backdoor Federated Learning
• Extended Isolation Forest
• Spectral Signatures in Backdoor Attacks
• The Extreme Value Machine
• Defensive Distillation is Not Robust to Adversarial Examples
• TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems
• Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong
• Out-of-distribution Detection in Classifiers via Generation
• Quantum Entropy Scoring for Fast Robust Mean Estimation and Improved Outlier Detection