Global Aggregation then Local Distribution for Scene Parsing
arXiv:2107.13154 · doi:10.1109/TIP.2021.3099366
Abstract
Modelling long-range contextual relationships is critical for pixel-wise prediction tasks such as semantic segmentation. However, convolutional neural networks (CNNs) are inherently limited to model such dependencies due to the naive structure in its building modules (\eg, local convolution kernel). While recent global aggregation methods are beneficial for long-range structure information modelling, they would oversmooth and bring noise to the regions containing fine details (\eg,~boundaries and small objects), which are very much cared for the semantic segmentation task. To alleviate this problem, we propose to explore the local context for making the aggregated long-range relationship being distributed more accurately in local regions. In particular, we design a novel local distribution module which models the affinity map between global and local relationship for each pixel adaptively. Integrating existing global aggregation modules, we show that our approach can be modularized as an end-to-end trainable block and easily plugged into existing semantic segmentation networks, giving rise to the \emph{GALD} networks. Despite its simplicity and versatility, our approach allows us to build new state of the art on major semantic segmentation benchmarks including Cityscapes, ADE20K, Pascal Context, Camvid and COCO-stuff. Code and trained models are released at \url{https://github.com/lxtGH/GALD-DGCNet} to foster further research.
arXiv commentAccepted by IEEE-TIP-2021. arXiv admin note: text overlap with arXiv:1909.07229
References in corpus (14)
- Rethinking Atrous Convolution for Semantic Image Segmentation
- Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
- Fully Convolutional Networks for Semantic Segmentation
- Conditional Random Fields as Recurrent Neural Networks
- ParseNet: Looking Wider to See Better
- Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
- Object Detectors Emerge in Deep Scene CNNs
- OCNet: Object Context Network for Scene Parsing
- Searching for Efficient Multi-Scale Architectures for Dense Image Prediction
- Semantic Understanding of Scenes through the ADE20K Dataset
- Compact Generalized Non-local Network
- Dual Graph Convolutional Network for Semantic Segmentation
- Global Aggregation then Local Distribution in Fully Convolutional Networks
- GFF: Gated Fully Fusion for Semantic Segmentation