Competitive Inner-Imaging Squeeze and Excitation for Residual Network

Abstract

Residual networks, which use a residual unit to supplement the identity mappings, enable very deep convolutional architecture to operate well, however, the residual architecture has been proved to be diverse and redundant, which may leads to low-efficient modeling. In this work, we propose a competitive squeeze-excitation (SE) mechanism for the residual network. Re-scaling the value for each channel in this structure will be determined by the residual and identity mappings jointly, and this design enables us to expand the meaning of channel relationship modeling in residual blocks. Modeling of the competition between residual and identity mappings cause the identity flow to control the complement of the residual feature maps for itself. Furthermore, we design a novel inner-imaging competitive SE block to shrink the consumption and re-image the global features of intermediate network structure, by using the inner-imaging mechanism, we can model the channel-wise relations with convolution in spatial. We carry out experiments on the CIFAR, SVHN, and ImageNet datasets, and the proposed method can challenge state-of-the-art results.

arXiv commentCode is available at https://github.com/scut-aitcm/Competitive-Inner-Imaging-SENet

References in corpus (7)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Neural Ordinary Differential Equations
• Interleaved Group Convolutions for Deep Neural Networks
• Reversible Architectures for Arbitrarily Deep Residual Neural Networks
• SkipNet: Learning Dynamic Routing in Convolutional Networks
• Dual Path Networks
• Connectivity Learning in Multi-Branch Networks

Cited by in corpus (6)

• Global Second-order Pooling Convolutional Networks
• Lightweight Residual Densely Connected Convolutional Neural Network
• ST-MTL: Spatio-Temporal Multitask Learning Model to Predict Scanpath While Tracking Instruments in Robotic Surgery
• Restoring Images with Unknown Degradation Factors by Recurrent Use of a Multi-branch Network
• Stochastic Region Pooling: Make Attention More Expressive
• Glioblastoma Multiforme Prognosis: MRI Missing Modality Generation, Segmentation and Radiogenomic Survival Prediction