Compact Generalized Non-local Network

Abstract

The non-local module is designed for capturing long-range spatio-temporal dependencies in images and videos. Although having shown excellent performance, it lacks the mechanism to model the interactions between positions across channels, which are of vital importance in recognizing fine-grained objects and actions. To address this limitation, we generalize the non-local module and take the correlations between the positions of any two channels into account. This extension utilizes the compact representation for multiple kernel functions with Taylor expansion that makes the generalized non-local module in a fast and low-complexity computation flow. Moreover, we implement our generalized non-local method within channel groups to ease the optimization. Experimental results illustrate the clear-cut improvements and practical applicability of the generalized non-local module on both fine-grained object recognition and video classification. Code is available at: https://github.com/KaiyuYue/cgnl-network.pytorch.

Technical report; To appear at NIPS 2018; Code is available at https://github.com/KaiyuYue/cgnl-network.pytorch

References in corpus (5)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
• Non-local Neural Networks

Cited by in corpus (25)

• A Survey on Visual Transformer
• Attention Mechanisms in Computer Vision: A Survey
• OCNet: Object Context Network for Scene Parsing
• Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation
• Second-order Non-local Attention Networks for Person Re-identification
• Image Compressed Sensing Using Non-local Neural Network
• Noise Conscious Training of Non Local Neural Network powered by Self Attentive Spectral Normalized Markovian Patch GAN for Low Dose CT Denoising
• Efficient Attention: Attention with Linear Complexities
• Accurate and Lightweight Image Super-Resolution with Model-Guided Deep Unfolding Network
• Interlaced Sparse Self-Attention for Semantic Segmentation
• Dynamic Neural Networks: A Survey
• Thickened 2D Networks for Efficient 3D Medical Image Segmentation
• Deep Learning based Data Prefetching in CPU-GPU Unified Virtual Memory
• Global Aggregation then Local Distribution for Scene Parsing
• Global Self-Attention Networks for Image Recognition
• Hyper-Convolution Networks for Biomedical Image Segmentation
• GTA: Global Temporal Attention for Video Action Understanding
• Compact Global Descriptor for Neural Networks
• Efficient Coarse-to-Fine Non-Local Module for the Detection of Small Objects
• Understanding Teacher Gaze Patterns for Robot Learning
• Real-time Semantic Segmentation with Fast Attention
• Region-based Non-local Operation for Video Classification
• Channel Interaction Networks for Fine-Grained Image Categorization
• Local Temporal Bilinear Pooling for Fine-grained Action Parsing
• DSANet: Dynamic Segment Aggregation Network for Video-Level Representation Learning