CaseNet: Content-Adaptive Scale Interaction Networks for Scene Parsing

Abstract

Objects at different spatial positions in an image exhibit different scales. Adaptive receptive fields are expected to capture suitable ranges of context for accurate pixel level semantic prediction. Recently, atrous convolution with different dilation rates has been used to generate features of multi-scales through several branches which are then fused for prediction. However, there is a lack of explicit interaction among the branches of different scales to adaptively make full use of the contexts. In this paper, we propose a Content-Adaptive Scale Interaction Network (CASINet) to exploit the multi-scale features for scene parsing. We build CASINet based on the classic Atrous Spatial Pyramid Pooling (ASPP) module, followed by a proposed contextual scale interaction (CSI) module, and a scale adaptation (SA) module. Specifically, in the CSI module, for each spatial position of some scale, instead of being limited by a fixed set of convolutional filters that are shared across different spatial positions for feature learning, we promote the adaptivity of the convolutional filters to spatial positions. We achieve this by the context interaction among the features of different scales. The SA module explicitly and softly selects the suitable scale for each spatial position and each channel. Ablation studies demonstrate the effectiveness of the proposed modules. We achieve state-of-the-art performance on three scene parsing benchmarks Cityscapes, ADE20K and LIP.

arXiv commentAccepted by Neurocomputing

References in corpus (9)

• Rethinking Atrous Convolution for Semantic Image Segmentation
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks
• High-Resolution Representations for Labeling Pixels and Regions
• OCNet: Object Context Network for Scene Parsing
• Context-Aware Visual Policy Network for Sequence-Level Image Captioning
• High-performance Semantic Segmentation Using Very Deep Fully Convolutional Networks
• Devil in the Details: Towards Accurate Single and Multiple Human Parsing
• Global Aggregation then Local Distribution in Fully Convolutional Networks

Cited by in corpus (2)

• Spatial--spectral FFPNet: Attention-Based Pyramid Network for Segmentation and Classification of Remote Sensing Images
• Scale Aware Adaptation for Land-Cover Classification in Remote Sensing Imagery