Spatial Group-wise Enhance: Improving Semantic Feature Learning in Convolutional Networks

Abstract

The Convolutional Neural Networks (CNNs) generate the feature representation of complex objects by collecting hierarchical and different parts of semantic sub-features. These sub-features can usually be distributed in grouped form in the feature vector of each layer, representing various semantic entities. However, the activation of these sub-features is often spatially affected by similar patterns and noisy backgrounds, resulting in erroneous localization and identification. We propose a Spatial Group-wise Enhance (SGE) module that can adjust the importance of each sub-feature by generating an attention factor for each spatial location in each semantic group, so that every individual group can autonomously enhance its learnt expression and suppress possible noise. The attention factors are only guided by the similarities between the global and local feature descriptors inside each group, thus the design of SGE module is extremely lightweight with \emph{almost no extra parameters and calculations}. Despite being trained with only category supervisions, the SGE component is extremely effective in highlighting multiple active areas with various high-order semantics (such as the dog's eyes, nose, etc.). When integrated with popular CNN backbones, SGE can significantly boost the performance of image recognition tasks. Specifically, based on ResNet50 backbones, SGE achieves 1.2\% Top-1 accuracy improvement on the ImageNet benchmark and 1.0$\sim$2.0\% AP gain on the COCO benchmark across a wide range of detectors (Faster/Mask/Cascade RCNN and RetinaNet). Codes and pretrained models are available at https://github.com/implus/PytorchInsight.

Code available at: https://github.com/implus/PytorchInsight

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Res2Net: A New Multi-scale Backbone Architecture
• MobileNetV2: Inverted Residuals and Linear Bottlenecks
• Bidirectional Attention Flow for Machine Comprehension
• Micro-Batch Training with Batch-Channel Normalization and Weight Standardization

Cited by in corpus (16)

• Efficient Multi-Scale Attention Module with Cross-Spatial Learning
• Salient Object Detection in Optical Remote Sensing Images Driven by Transformer
• Channel Distillation: Channel-Wise Attention for Knowledge Distillation
• TEINet: Towards an Efficient Architecture for Video Recognition
• Detection and Tracking Meet Drones Challenge
• Convolution-Weight-Distribution Assumption: Rethinking the Criteria of Channel Pruning
• Efficient Attention Network: Accelerate Attention by Searching Where to Plug
• DCANet: Learning Connected Attentions for Convolutional Neural Networks
• Instance Enhancement Batch Normalization: an Adaptive Regulator of Batch Noise
• Rega-Net:Retina Gabor Attention for Deep Convolutional Neural Networks
• BA^2M: A Batch Aware Attention Module for Image Classification
• Spectral Unsupervised Domain Adaptation for Visual Recognition
• Understanding the Disharmony between Weight Normalization Family and Weight Decay: $\epsilon -$shifted $L_2$ Regularizer
• UPI-Net: Semantic Contour Detection in Placental Ultrasound
• Channel Pruning Guided by Spatial and Channel Attention for DNNs in Intelligent Edge Computing
• Poly-NL: Linear Complexity Non-local Layers with Polynomials