Understanding Intra-Class Knowledge Inside CNN

Abstract

Convolutional Neural Network (CNN) has been successful in image recognition tasks, and recent works shed lights on how CNN separates different classes with the learned inter-class knowledge through visualization. In this work, we instead visualize the intra-class knowledge inside CNN to better understand how an object class is represented in the fully-connected layers. To invert the intra-class knowledge into more interpretable images, we propose a non-parametric patch prior upon previous CNN visualization models. With it, we show how different "styles" of templates for an object class are organized by CNN in terms of location and content, and represented in a hierarchical and ensemble way. Moreover, such intra-class knowledge can be used in many interesting applications, e.g. style-based image retrieval and style-based object completion.

tech report for: http://vision03.csail.mit.edu/cnn_art/index.html

References in corpus (1)

• Do Convnets Learn Correspondence?

Cited by in corpus (22)

• Interpretable machine learning: definitions, methods, and applications
• What makes ImageNet good for transfer learning?
• Synthesizing the preferred inputs for neurons in neural networks via deep generator networks
• Do Convolutional Neural Networks Learn Class Hierarchy?
• Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks
• How convolutional neural network see the world - A survey of convolutional neural network visualization methods
• Beyond saliency: understanding convolutional neural networks from saliency prediction on layer-wise relevance propagation
• Explainable Machine Learning with Prior Knowledge: An Overview
• Sparse Oblique Decision Trees: A Tool to Understand and Manipulate Neural Net Features
• How Do Neural Networks Estimate Optical Flow? A Neuropsychology-Inspired Study
• Towards Better Analysis of Deep Convolutional Neural Networks
• Semantics for Global and Local Interpretation of Deep Neural Networks
• On monitoring development indicators using high resolution satellite images
• Attacking Convolutional Neural Network using Differential Evolution
• Diverse feature visualizations reveal invariances in early layers of deep neural networks
• Sampling the "Inverse Set" of a Neuron: An Approach to Understanding Neural Nets
• Class Subset Selection for Transfer Learning using Submodularity
• Exploring the Deep Feature Space of a Cell Classification Neural Network
• Every Filter Extracts A Specific Texture In Convolutional Neural Networks
• Low-Cost Transfer Learning of Face Tasks
• Towards glass-box CNNs
• On the Selective and Invariant Representation of DCNN for High-Resolution Remote Sensing Image Recognition