Topological Detection of Trojaned Neural Networks
arXiv:2106.06469
Abstract
Deep neural networks are known to have security issues. One particular threat is the Trojan attack. It occurs when the attackers stealthily manipulate the model's behavior through Trojaned training samples, which can later be exploited. Guided by basic neuroscientific principles we discover subtle -- yet critical -- structural deviation characterizing Trojaned models. In our analysis we use topological tools. They allow us to model high-order dependencies in the networks, robustly compare different networks, and localize structural abnormalities. One interesting observation is that Trojaned models develop short-cuts from input to output layers. Inspired by these observations, we devise a strategy for robust detection of Trojaned models. Compared to standard baselines it displays better performance on multiple benchmarks.
References in corpus (15)
- Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
- BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
- Poisoning Attacks against Support Vector Machines
- Statistical topological data analysis using persistence landscapes
- Confidence sets for persistence diagrams
- Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering
- TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems
- Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation
- Sliced Wasserstein Kernel for Persistence Diagrams
- A Topological Regularizer for Classifiers via Persistent Homology
- Weight Poisoning Attacks on Pre-trained Models
- Connectivity-Optimized Representation Learning via Persistent Homology
- TrojDRL: Trojan Attacks on Deep Reinforcement Learning Agents
- Towards Interaction Detection Using Topological Analysis on Neural Networks
- TopoTxR: A Topological Biomarker for Predicting Treatment Response in Breast Cancer