ORDNet: Capturing Omni-Range Dependencies for Scene Parsing
arXiv:2101.03929 · doi:10.1109/TIP.2020.3013142
Abstract
Learning to capture dependencies between spatial positions is essential to many visual tasks, especially the dense labeling problems like scene parsing. Existing methods can effectively capture long-range dependencies with self-attention mechanism while short ones by local convolution. However, there is still much gap between long-range and short-range dependencies, which largely reduces the models' flexibility in application to diverse spatial scales and relationships in complicated natural scene images. To fill such a gap, we develop a Middle-Range (MR) branch to capture middle-range dependencies by restricting self-attention into local patches. Also, we observe that the spatial regions which have large correlations with others can be emphasized to exploit long-range dependencies more accurately, and thus propose a Reweighed Long-Range (RLR) branch. Based on the proposed MR and RLR branches, we build an Omni-Range Dependencies Network (ORDNet) which can effectively capture short-, middle- and long-range dependencies. Our ORDNet is able to extract more comprehensive context information and well adapt to complex spatial variance in scene images. Extensive experiments show that our proposed ORDNet outperforms previous state-of-the-art methods on three scene parsing benchmarks including PASCAL Context, COCO Stuff and ADE20K, demonstrating the superiority of capturing omni-range dependencies in deep models for scene parsing task.
Published at TIP
References in corpus (19)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Neural Machine Translation by Jointly Learning to Align and Translate
- Rethinking Atrous Convolution for Semantic Image Segmentation
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
- Fully Convolutional Networks for Semantic Segmentation
- Self-Attention Generative Adversarial Networks
- Conditional Random Fields as Recurrent Neural Networks
- Squeeze-and-Excitation Networks
- Hierarchical Question-Image Co-Attention for Visual Question Answering
- ParseNet: Looking Wider to See Better
- Wider or Deeper: Revisiting the ResNet Model for Visual Recognition
- Bridging Category-level and Instance-level Semantic Image Segmentation
- Efficient piecewise training of deep structured models for semantic segmentation
- PixelNet: Towards a General Pixel-level Architecture
- Interlaced Sparse Self-Attention for Semantic Segmentation
- LabelBank: Revisiting Global Perspectives for Semantic Segmentation
- Top-Down Learning for Structured Labeling with Convolutional Pseudoprior