Scientific Calculator for Designing Trojan Detectors in Neural Networks
arXiv:2006.03707
Abstract
This work presents a web-based interactive neural network (NN) calculator and a NN inefficiency measurement that has been investigated for the purpose of detecting trojans embedded in NN models. This NN Calculator is designed on top of TensorFlow Playground with in-memory storage of data and NN graphs plus coefficients. It is "like a scientific calculator" with analytical, visualization, and output operations performed on training datasets and NN architectures. The prototype is aaccessible at https://pages.nist.gov/nn-calculator. The analytical capabilities include a novel measurement of NN inefficiency using modified Kullback-Liebler (KL) divergence applied to histograms of NN model states, as well as a quantification of the sensitivity to variables related to data and NNs. Both NN Calculator and KL divergence are used to devise a trojan detector approach for a variety of trojan embeddings. Experimental results document desirable properties of the KL divergence measurement with respect to NN architectures and dataset perturbations, as well as inferences about embedded trojans.
Presented at AAAI FSS-20: Artificial Intelligence in Government and Public Sector, Washington, DC, USA
References in corpus (7)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Deep Learning with Limited Numerical Precision
- Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures
- What Does Explainable AI Really Mean? A New Conceptualization of Perspectives
- Direct-Manipulation Visualization of Deep Networks
- Mathematics of Deep Learning
- Modeling Order in Neural Word Embeddings at Scale