Object-ABN: Learning to Generate Sharp Attention Maps for Action Recognition
arXiv:2207.13306 · doi:10.1587/transinf.2022EDP7138
Abstract
In this paper we propose an extension of the Attention Branch Network (ABN) by using instance segmentation for generating sharper attention maps for action recognition. Methods for visual explanation such as Grad-CAM usually generate blurry maps which are not intuitive for humans to understand, particularly in recognizing actions of people in videos. Our proposed method, Object-ABN, tackles this issue by introducing a new mask loss that makes the generated attention maps close to the instance segmentation result. Further the PC loss and multiple attention maps are introduced to enhance the sharpness of the maps and improve the performance of classification. Experimental results with UCF101 and SSv2 shows that the generated maps by the proposed method are much clearer qualitatively and quantitatively than those of the original ABN.
arXiv comment9 pages
References in corpus (17)
- Adam: A Method for Stochastic Optimization
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
- The Kinetics Human Action Video Dataset
- A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI
- Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
- Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey
- Action Recognition using Visual Attention
- Attentional Pooling for Action Recognition
- A Comprehensive Study of Deep Video Action Recognition
- Saliency Tubes: Visual Explanations for Spatio-Temporal Convolutions
- Video Action Understanding
- GTA: Global Temporal Attention for Video Action Understanding
- Class Feature Pyramids for Video Explanation
- Spatio-Temporal Perturbations for Video Attribution
- Explainable Deep Learning for Video Recognition Tasks: A Framework & Recommendations
- Discriminating Spatial and Temporal Relevance in Deep Taylor Decompositions for Explainable Activity Recognition