How convolutional neural network see the world - A survey of convolutional neural network visualization methods

Abstract

Nowadays, the Convolutional Neural Networks (CNNs) have achieved impressive performance on many computer vision related tasks, such as object detection, image recognition, image retrieval, etc. These achievements benefit from the CNNs outstanding capability to learn the input features with deep layers of neuron structures and iterative training process. However, these learned features are hard to identify and interpret from a human vision perspective, causing a lack of understanding of the CNNs internal working mechanism. To improve the CNN interpretability, the CNN visualization is well utilized as a qualitative analysis method, which translates the internal features into visually perceptible patterns. And many CNN visualization works have been proposed in the literature to interpret the CNN in perspectives of network structure, operation, and semantic concept. In this paper, we expect to provide a comprehensive survey of several representative CNN visualization methods, including Activation Maximization, Network Inversion, Deconvolutional Neural Networks (DeconvNet), and Network Dissection based visualization. These methods are presented in terms of motivations, algorithms, and experiment results. Based on these visualization methods, we also discuss their practical applications to demonstrate the significance of the CNN interpretability in areas of network design, optimization, security enhancement, etc.

32 pages, 21 figures. Mathematical Foundations of Computing

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Understanding Neural Networks Through Deep Visualization
• Object Detectors Emerge in Deep Scene CNNs
• Visualizing Deep Convolutional Neural Networks Using Natural Pre-Images
• Synthesizing the preferred inputs for neurons in neural networks via deep generator networks
• Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks

Cited by in corpus (7)

• DASS: Differentiable Architecture Search for Sparse neural networks
• Single Unit Status in Deep Convolutional Neural Network Codes for Face Identification: Sparseness Redefined
• On the Evaluation of the Plausibility and Faithfulness of Sentiment Analysis Explanations
• A new way to constrain the densities of intra-group medium in groups of galaxies with convolutional neural networks
• Embedded Encoder-Decoder in Convolutional Networks Towards Explainable AI
• A Tour of Convolutional Networks Guided by Linear Interpreters
• Separating Content and Style for Unsupervised Image-to-Image Translation