Learn To Pay Attention

Abstract

We propose an end-to-end-trainable attention module for convolutional neural network (CNN) architectures built for image classification. The module takes as input the 2D feature vector maps which form the intermediate representations of the input image at different stages in the CNN pipeline, and outputs a 2D matrix of scores for each map. Standard CNN architectures are modified through the incorporation of this module, and trained under the constraint that a convex combination of the intermediate 2D feature vectors, as parameterised by the score matrices, must \textit{alone} be used for classification. Incentivised to amplify the relevant and suppress the irrelevant or misleading, the scores thus assume the role of attention values. Our experimental observations provide clear evidence to this effect: the learned attention maps neatly highlight the regions of interest while suppressing background clutter. Consequently, the proposed function is able to bootstrap standard CNN architectures for the task of image classification, demonstrating superior generalisation over 6 unseen benchmark datasets. When binarised, our attention maps outperform other CNN-based attention maps, traditional saliency maps, and top object proposals for weakly supervised segmentation as demonstrated on the Object Discovery dataset. We also demonstrate improved robustness against the fast gradient sign method of adversarial attack.

arXiv commentInternational Conference on Learning Representations 2018

References in corpus (3)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Recurrent Models of Visual Attention

Cited by in corpus (42)

• Attention U-Net: Learning Where to Look for the Pancreas
• Diagnosis of Alzheimer's Disease via Multi-modality 3D Convolutional Neural Network
• Attention-based Convolutional Neural Network for Weakly Labeled Human Activities Recognition with Wearable Sensors
• Multimodal Spatial Attention Module for Targeting Multimodal PET-CT Lung Tumor Segmentation
• FragNet: Writer Identification using Deep Fragment Networks
• Sequential vessel segmentation via deep channel attention network
• Sequential Weakly Labeled Multi-Activity Localization and Recognition on Wearable Sensors using Recurrent Attention Networks
• Adversarial Explanations for Understanding Image Classification Decisions and Improved Neural Network Robustness
• SAUNet: Shape Attentive U-Net for Interpretable Medical Image Segmentation
• Attention Convolutional Binary Neural Tree for Fine-Grained Visual Categorization
• Knowledge Squeezed Adversarial Network Compression
• L2AE-D: Learning to Aggregate Embeddings for Few-shot Learning with Meta-level Dropout
• FREA-Unet: Frequency-aware U-net for Modality Transfer
• Gated Hierarchical Attention for Image Captioning
• PDANet: Pyramid Density-aware Attention Net for Accurate Crowd Counting
• Achievements and Challenges in Explaining Deep Learning based Computer-Aided Diagnosis Systems
• Automatic Tracking of the Muscle Tendon Junction in Healthy and Impaired Subjects using Deep Learning
• Recurrent Feedback Improves Feedforward Representations in Deep Neural Networks
• Feature Prioritization and Regularization Improve Standard Accuracy and Adversarial Robustness
• Deep Fusion of Local and Non-Local Features for Precision Landslide Recognition
• The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks
• Passive Attention in Artificial Neural Networks Predicts Human Visual Selectivity
• Deep Discriminative Representation Learning with Attention Map for Scene Classification
• E-PixelHop: An Enhanced PixelHop Method for Object Classification
• State-of-the-art Techniques in Deep Edge Intelligence
• CNNs, LSTMs, and Attention Networks for Pathology Detection in Medical Data
• See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networks
• Collaborative Multi-agent Learning for MR Knee Articular Cartilage Segmentation
• Bounded logit attention: Learning to explain image classifiers
• Sharpen Focus: Learning with Attention Separability and Consistency
• Semantic-guided Automatic Natural Image Matting with Trimap Generation Network and Light-weight Non-local Attention
• PneumoXttention: A CNN compensating for Human Fallibility when Detecting Pneumonia through CXR images with Attention
• Reproduction Report on "Learn to Pay Attention"
• Revealing Lung Affections from CTs. A Comparative Analysis of Various Deep Learning Approaches for Dealing with Volumetric Data
• Attend and Rectify: a Gated Attention Mechanism for Fine-Grained Recovery
• Tissue characterization based on the analysis on i3DUS data for diagnosis support in neurosurgery
• A Weakly-Supervised Semantic Segmentation Approach based on the Centroid Loss: Application to Quality Control and Inspection
• Dual Normalization Multitasking for Audio-Visual Sounding Object Localization
• Careful analysis of XRD patterns with Attention
• On Improving the Generalization of Face Recognition in the Presence of Occlusions
• LIRA: Lifelong Image Restoration from Unknown Blended Distortions
• Attending Category Disentangled Global Context for Image Classification