Tell Me Where to Look: Guided Attention Inference Network

Abstract

Weakly supervised learning with only coarse labels can obtain visual explanations of deep neural network such as attention maps by back-propagating gradients. These attention maps are then available as priors for tasks such as object localization and semantic segmentation. In one common framework we address three shortcomings of previous approaches in modeling such attention maps: We (1) first time make attention maps an explicit and natural component of the end-to-end training, (2) provide self-guidance directly on these maps by exploring supervision form the network itself to improve them, and (3) seamlessly bridge the gap between using weak and extra supervision if available. Despite its simplicity, experiments on the semantic segmentation task demonstrate the effectiveness of our methods. We clearly surpass the state-of-the-art on Pascal VOC 2012 val. and test set. Besides, the proposed framework provides a way not only explaining the focus of the learner but also feeding back with direct guidance towards specific tasks. Under mild assumptions our method can also be understood as a plug-in to existing weakly supervised learners to improve their generalization performance.

arXiv commentAccepted in CVPR2018

References in corpus (9)

• Caffe: Convolutional Architecture for Fast Feature Embedding
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
• Object Detectors Emerge in Deep Scene CNNs
• STC: A Simple to Complex Framework for Weakly-supervised Semantic Segmentation
• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
• Image De-raining Using a Conditional Generative Adversarial Network
• MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Network
• Built-in Foreground/Background Prior for Weakly-Supervised Semantic Segmentation
• Learning Compositional Visual Concepts with Mutual Consistency

Cited by in corpus (19)

• Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks
• Residual Dense Network for Image Super-Resolution
• Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks
• Residual Dense Network for Image Restoration
• Deep Weakly-Supervised Learning Methods for Classification and Localization in Histology Images: A Survey
• Evaluating Explanation Without Ground Truth in Interpretable Machine Learning
• Self-Erasing Network for Integral Object Attention
• Looking in the Right place for Anomalies: Explainable AI through Automatic Location Learning
• Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast
• GATCluster: Self-Supervised Gaussian-Attention Network for Image Clustering
• Automatic Diagnosis of Pulmonary Embolism Using an Attention-guided Framework: A Large-scale Study
• Class Activation Map Generation by Representative Class Selection and Multi-Layer Feature Fusion
• Learning Compositional Visual Concepts with Mutual Consistency
• Improved Techniques For Weakly-Supervised Object Localization
• Multi-Miner: Object-Adaptive Region Mining for Weakly-Supervised Semantic Segmentation
• DeepOpht: Medical Report Generation for Retinal Images via Deep Models and Visual Explanation
• Leveraging Explainability for Comprehending Referring Expressions in the Real World
• SuperMeshing: A New Deep Learning Architecture for Increasing the Mesh Density of Metal Forming Stress Field with Attention Mechanism and Perceptual Features
• GANtron: Emotional Speech Synthesis with Generative Adversarial Networks