Explainable Adversarial Attacks in Deep Neural Networks Using Activation Profiles

Abstract

As neural networks become the tool of choice to solve an increasing variety of problems in our society, adversarial attacks become critical. The possibility of generating data instances deliberately designed to fool a network's analysis can have disastrous consequences. Recent work has shown that commonly used methods for model training often result in fragile abstract representations that are particularly vulnerable to such attacks. This paper presents a visual framework to investigate neural network models subjected to adversarial examples, revealing how models' perception of the adversarial data differs from regular data instances and their relationships with class perception. Through different use cases, we show how observing these elements can quickly pinpoint exploited areas in a model, allowing further study of vulnerable features in input data and serving as a guide to improving model training and architecture.

Submitted to EUROVIS 2021

References in corpus (16)

• UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
• A Survey of Deep Learning Techniques for Autonomous Driving
• Understanding Neural Networks Through Deep Visualization
• Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• Visualizing and Understanding Recurrent Networks
• Adversarial Attacks and Defences: A Survey
• CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualization
• RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records
• Generating Adversarial Examples with Adversarial Networks
• Manifold: A Model-Agnostic Framework for Interpretation and Diagnosis of Machine Learning Models
• Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks
• Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation
• Explaining Vulnerabilities to Adversarial Machine Learning through Visual Analytics
• Robust Neural Networks using Randomized Adversarial Training
• An Introduction to Deep Visual Explanation