AlignSeg: Feature-Aligned Segmentation Networks
arXiv:2003.00872
Abstract
Aggregating features in terms of different convolutional blocks or contextual embeddings has been proven to be an effective way to strengthen feature representations for semantic segmentation. However, most of the current popular network architectures tend to ignore the misalignment issues during the feature aggregation process caused by 1) step-by-step downsampling operations, and 2) indiscriminate contextual information fusion. In this paper, we explore the principles in addressing such feature misalignment issues and inventively propose Feature-Aligned Segmentation Networks (AlignSeg). AlignSeg consists of two primary modules, i.e., the Aligned Feature Aggregation (AlignFA) module and the Aligned Context Modeling (AlignCM) module. First, AlignFA adopts a simple learnable interpolation strategy to learn transformation offsets of pixels, which can effectively relieve the feature misalignment issue caused by multiresolution feature aggregation. Second, with the contextual embeddings in hand, AlignCM enables each pixel to choose private custom contextual information in an adaptive manner, making the contextual embeddings aligned better to provide appropriate guidance. We validate the effectiveness of our AlignSeg network with extensive experiments on Cityscapes and ADE20K, achieving new state-of-the-art mIoU scores of 82.6% and 45.95%, respectively. Our source code will be made available.
Accepted by TPAMI 2021
References in corpus (5)
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Rethinking Atrous Convolution for Semantic Image Segmentation
- OCNet: Object Context Network for Scene Parsing
- Searching for Efficient Multi-Scale Architectures for Dense Image Prediction
- Revisiting Feature Alignment for One-stage Object Detection
Cited by in corpus (4)
- Half-Real Half-Fake Distillation for Class-Incremental Semantic Segmentation
- The 1st Agriculture-Vision Challenge: Methods and Results
- AttaNet: Attention-Augmented Network for Fast and Accurate Scene Parsing
- Alleviating Semantic-level Shift: A Semi-supervised Domain Adaptation Method for Semantic Segmentation