Compact Global Descriptor for Neural Networks
arXiv:1907.09665
Abstract
Long-range dependencies modeling, widely used in capturing spatiotemporal correlation, has shown to be effective in CNN dominated computer vision tasks. Yet neither stacks of convolutional operations to enlarge receptive fields nor recent nonlocal modules is computationally efficient. In this paper, we present a generic family of lightweight global descriptors for modeling the interactions between positions across different dimensions (e.g., channels, frames). This descriptor enables subsequent convolutions to access the informative global features with negligible computational complexity and parameters. Benchmark experiments show that the proposed method can complete state-of-the-art long-range mechanisms with a significant reduction in extra computing cost. Code available at https://github.com/HolmesShuan/Compact-Global-Descriptor.
arXiv commentThis paper will be included in our future works as a subsection
References in corpus (24)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Rethinking Atrous Convolution for Semantic Image Segmentation
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
- Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
- UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
- Wide Residual Networks
- Multi-Scale Context Aggregation by Dilated Convolutions
- Do Deep Nets Really Need to be Deep?
- Rethinking the Value of Network Pruning
- Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
- Rethinking the Inception Architecture for Computer Vision
- A simple neural network module for relational reasoning
- Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks
- Deformable Convolutional Networks
- Channel Pruning for Accelerating Very Deep Neural Networks
- CBAM: Convolutional Block Attention Module
- ABC-CNN: An Attention Based Convolutional Neural Network for Visual Question Answering
- Stacked Attention Networks for Image Question Answering
- Discrimination-aware Channel Pruning for Deep Neural Networks
- Compact Generalized Non-local Network
- BAM: Bottleneck Attention Module
- Nonlocal Neural Networks, Nonlocal Diffusion and Nonlocal Modeling