Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Cases

Abstract

When the training data are maliciously tampered, the predictions of the acquired deep neural network (DNN) can be manipulated by an adversary known as the Trojan attack (or poisoning backdoor attack). The lack of robustness of DNNs against Trojan attacks could significantly harm real-life machine learning (ML) systems in downstream applications, therefore posing widespread concern to their trustworthiness. In this paper, we study the problem of the Trojan network (TrojanNet) detection in the data-scarce regime, where only the weights of a trained DNN are accessed by the detector. We first propose a data-limited TrojanNet detector (TND), when only a few data samples are available for TrojanNet detection. We show that an effective data-limited TND can be established by exploring connections between Trojan attack and prediction-evasion adversarial attacks including per-sample attack as well as all-sample universal attack. In addition, we propose a data-free TND, which can detect a TrojanNet without accessing any data samples. We show that such a TND can be built by leveraging the internal response of hidden neurons, which exhibits the Trojan behavior even at random noise inputs. The effectiveness of our proposals is evaluated by extensive experiments under different model architectures and datasets including CIFAR-10, GTSRB, and ImageNet.

References in corpus (11)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Certified Adversarial Robustness via Randomized Smoothing
• Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering
• Transferable Clean-Label Poisoning Attacks on Deep Neural Nets
• TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems
• Learning with Bad Training Data via Iterative Trimmed Loss Minimization
• Defending against Backdoor Attack on Deep Neural Networks
• Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness

Cited by in corpus (8)

• Poison Ink: Robust and Invisible Backdoor Attack
• Backdoor Scanning for Deep Neural Networks through K-Arm Optimization
• Poisoned classifiers are not only backdoored, they are fundamentally broken
• TOP: Backdoor Detection in Neural Networks via Transferability of Perturbation
• Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models
• EX-RAY: Distinguishing Injected Backdoor from Natural Features in Neural Networks by Examining Differential Feature Symmetry
• Two Sides of the Same Coin: Learning the Backdoor to Remove the Backdoor
• SGBA: A Stealthy Scapegoat Backdoor Attack against Deep Neural Networks