Learning what and where to attend
arXiv:1805.08819
Abstract
Most recent gains in visual recognition have originated from the inclusion of attention mechanisms in deep convolutional networks (DCNs). Because these networks are optimized for object recognition, they learn where to attend using only a weak form of supervision derived from image class labels. Here, we demonstrate the benefit of using stronger supervisory signals by teaching DCNs to attend to image regions that humans deem important for object recognition. We first describe a large-scale online experiment (ClickMe) used to supplement ImageNet with nearly half a million human-derived "top-down" attention maps. Using human psychophysics, we confirm that the identified top-down features from ClickMe are more diagnostic than "bottom-up" saliency features for rapid image categorization. As a proof of concept, we extend a state-of-the-art attention network and demonstrate that adding ClickMe supervision significantly improves its accuracy and yields visual features that are more interpretable and more similar to those used by human observers.
arXiv commentPreviously called Global-and-local attention networks for visual recognition. Current version published in ICLR 2019: https://openreview.net/forum?id=BJgLg3R9KQ
Cited by in corpus (5)
- Attention Mechanisms in Computer Vision: A Survey
- Coordinate Attention for Efficient Mobile Network Design
- Rotate to Attend: Convolutional Triplet Attention Module
- Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysis
- Explainability via Interactivity? Supporting Nonexperts' Sensemaking of Pretrained CNN by Interacting with Their Daily Surroundings