LatentGNN: Learning Efficient Non-local Relations for Visual Recognition

Abstract

Capturing long-range dependencies in feature representations is crucial for many visual recognition tasks. Despite recent successes of deep convolutional networks, it remains challenging to model non-local context relations between visual features. A promising strategy is to model the feature context by a fully-connected graph neural network (GNN), which augments traditional convolutional features with an estimated non-local context representation. However, most GNN-based approaches require computing a dense graph affinity matrix and hence have difficulty in scaling up to tackle complex real-world visual problems. In this work, we propose an efficient and yet flexible non-local relation representation based on a novel class of graph neural networks. Our key idea is to introduce a latent space to reduce the complexity of graph, which allows us to use a low-rank representation for the graph affinity matrix and to achieve a linear complexity in computation. Extensive experimental evaluations on three major visual recognition tasks show that our method outperforms the prior works with a large margin while maintaining a low computation cost.

ICML 2019

References in corpus (8)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• Conditional Random Fields as Recurrent Neural Networks
• Molecular Graph Convolutions: Moving Beyond Fingerprints
• Deep Convolutional Networks on Graph-Structured Data
• Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes

Cited by in corpus (13)

• Visual Transformers: Token-based Image Representation and Processing for Computer Vision
• Pre-Trained Image Processing Transformer
• Is Attention Better Than Matrix Decomposition?
• Graph Neural Networks in Computer Vision -- Architectures, Datasets and Common Approaches
• Bipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generation
• Tensor Low-Rank Reconstruction for Semantic Segmentation
• Revisiting Spatial Invariance with Low-Rank Local Connectivity
• Cascaded Non-local Neural Network for Point Cloud Semantic Segmentation
• A^2-FPN: Attention Aggregation based Feature Pyramid Network for Instance Segmentation
• Discovering Dynamic Salient Regions for Spatio-Temporal Graph Neural Networks
• Learning Implicit Temporal Alignment for Few-shot Video Classification
• Cross-Modal Progressive Comprehension for Referring Segmentation
• Affinity-aware Compression and Expansion Network for Human Parsing