Neural Trojans

Abstract

While neural networks demonstrate stronger capabilities in pattern recognition nowadays, they are also becoming larger and deeper. As a result, the effort needed to train a network also increases dramatically. In many cases, it is more practical to use a neural network intellectual property (IP) that an IP vendor has already trained. As we do not know about the training process, there can be security threats in the neural IP: the IP vendor (attacker) may embed hidden malicious functionality, i.e. neural Trojans, into the neural IP. We show that this is an effective attack and provide three mitigation techniques: input anomaly detection, re-training, and input preprocessing. All the techniques are proven effective. The input anomaly detection approach is able to detect 99.8% of Trojan triggers although with 12.2% false positive. The re-training approach is able to prevent 94.1% of Trojan triggers from triggering the Trojan although it requires that the neural IP be reconfigurable. In the input preprocessing approach, 90.2% of Trojan triggers are rendered ineffective and no assumption about the neural IP is needed.

arXiv commentThe shorth-length version of this paper is to appear in the 2017 IEEE International Conference on Computer Design (ICCD)

References in corpus (4)

• Explaining and Harnessing Adversarial Examples
• Intriguing properties of neural networks
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Poisoning Attacks against Support Vector Machines

Cited by in corpus (18)

• Februus: Input Purification Defense Against Trojan Attacks on Deep Neural Network Systems
• Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning
• Robust Machine Learning Systems: Challenges, Current Trends, Perspectives, and the Road Ahead
• AI Security for Geoscience and Remote Sensing: Challenges and Future Trends
• Backdoor Attacks against Transfer Learning with Pre-trained Deep Learning Models
• Stop-and-Go: Exploring Backdoor Attacks on Deep Reinforcement Learning-based Traffic Congestion Control Systems
• Transferable Graph Backdoor Attack
• Federated and Transfer Learning: A Survey on Adversaries and Defense Mechanisms
• DeepiSign: Invisible Fragile Watermark to Protect the Integrityand Authenticity of CNN
• Active Poisoning: Efficient Backdoor Attacks on Transfer Learning-Based Brain-Computer Interfaces
• Embedding and Extraction of Knowledge in Tree Ensemble Classifiers
• Model Supply Chain Poisoning: Backdooring Pre-trained Models via Embedding Indistinguishability
• Steganography Beyond Space-Time with Chain of Multimodal AI
• Path Analysis for Effective Fault Localization in Deep Neural Networks
• BackdoorMBTI: A Backdoor Learning Multimodal Benchmark Tool Kit for Backdoor Defense Evaluation
• Dynamic Black-box Backdoor Attacks on IoT Sensory Data
• Imitation Game for Adversarial Disillusion with Chain-of-Thought Reasoning in Generative AI
• Hypnopaedia-Aware Machine Unlearning via Psychometrics of Artificial Mental Imagery