Selective Kernel Networks

Abstract

In standard Convolutional Neural Networks (CNNs), the receptive fields of artificial neurons in each layer are designed to share the same size. It is well-known in the neuroscience community that the receptive field size of visual cortical neurons are modulated by the stimulus, which has been rarely considered in constructing CNNs. We propose a dynamic selection mechanism in CNNs that allows each neuron to adaptively adjust its receptive field size based on multiple scales of input information. A building block called Selective Kernel (SK) unit is designed, in which multiple branches with different kernel sizes are fused using softmax attention that is guided by the information in these branches. Different attentions on these branches yield different sizes of the effective receptive fields of neurons in the fusion layer. Multiple SK units are stacked to a deep network termed Selective Kernel Networks (SKNets). On the ImageNet and CIFAR benchmarks, we empirically show that SKNet outperforms the existing state-of-the-art architectures with lower model complexity. Detailed analyses show that the neurons in SKNet can capture target objects with different scales, which verifies the capability of neurons for adaptively adjusting their receptive field sizes according to the input. The code and models are available at https://github.com/implus/SKNet.

CVPR 2019

References in corpus (12)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• FractalNet: Ultra-Deep Neural Networks without Residuals
• Deformable Convolutional Networks
• Residual Networks of Residual Networks: Multilevel Residual Networks
• CBAM: Convolutional Block Attention Module
• Shake-Shake regularization
• Residual Attention Network for Image Classification
• Image Super-Resolution Using Very Deep Residual Channel Attention Networks
• ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
• Multi-Residual Networks: Improving the Speed and Accuracy of Residual Networks
• IGCV$2$: Interleaved Structured Sparse Convolutional Neural Networks
• Person Search via A Mask-Guided Two-Stream CNN Model

Cited by in corpus (19)

• ABCNet: Attentive Bilateral Contextual Network for Efficient Semantic Segmentation of Fine-Resolution Remote Sensing Images
• You Only Learn One Representation: Unified Network for Multiple Tasks
• Autonomous Driving with Deep Learning: A Survey of State-of-Art Technologies
• TAda! Temporally-Adaptive Convolutions for Video Understanding
• Regular Polytope Networks
• Scaling-Translation-Equivariant Networks with Decomposed Convolutional Filters
• A Survey on Deep Domain Adaptation and Tiny Object Detection Challenges, Techniques and Datasets
• On the Integration of Self-Attention and Convolution
• SK-Unet: an Improved U-net Model with Selective Kernel for the Segmentation of Multi-sequence Cardiac MR
• Crowd Scene Analysis by Output Encoding
• DIANet: Dense-and-Implicit Attention Network
• Split to Be Slim: An Overlooked Redundancy in Vanilla Convolution
• Instance Enhancement Batch Normalization: an Adaptive Regulator of Batch Noise
• BA^2M: A Batch Aware Attention Module for Image Classification
• Cascade Attentive Dropout for Weakly Supervised Object Detection
• Glioblastoma Multiforme Prognosis: MRI Missing Modality Generation, Segmentation and Radiogenomic Survival Prediction
• LIRA: Lifelong Image Restoration from Unknown Blended Distortions
• 2nd Place Solution to Instance Segmentation of IJCAI 3D AI Challenge 2020
• Discovering Spatial Relationships by Transformers for Domain Generalization