Dense Prediction with Attentive Feature Aggregation
arXiv:2111.00770
Abstract
Aggregating information from features across different layers is an essential operation for dense prediction models. Despite its limited expressiveness, feature concatenation dominates the choice of aggregation operations. In this paper, we introduce Attentive Feature Aggregation (AFA) to fuse different network layers with more expressive non-linear operations. AFA exploits both spatial and channel attention to compute weighted average of the layer activations. Inspired by neural volume rendering, we extend AFA with Scale-Space Rendering (SSR) to perform late fusion of multi-scale predictions. AFA is applicable to a wide range of existing network designs. Our experiments show consistent and significant improvements on challenging semantic segmentation benchmarks, including Cityscapes, BDD100K, and Mapillary Vistas, at negligible computational and parameter overhead. In particular, AFA improves the performance of the Deep Layer Aggregation (DLA) model by nearly 6% mIoU on Cityscapes. Our experimental analyses show that AFA learns to progressively refine segmentation maps and to improve boundary details, leading to new state-of-the-art results on boundary detection benchmarks on BSDS500 and NYUDv2. Code and video resources are available at http://vis.xyz/pub/dla-afa.
20 pages, 14 figures, WACV 2023
References in corpus (15)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Rethinking Atrous Convolution for Semantic Image Segmentation
- Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
- The Cityscapes Dataset for Semantic Urban Scene Understanding
- Richer Convolutional Features for Edge Detection
- Hierarchical Multi-Scale Attention for Semantic Segmentation
- CBAM: Convolutional Block Attention Module
- Dilated Residual Networks
- DeeperLab: Single-Shot Image Parser
- Region Mutual Information Loss for Semantic Segmentation
- Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation
- Pixel Difference Networks for Efficient Edge Detection
- Learning to predict crisp boundaries
- SegFix: Model-Agnostic Boundary Refinement for Segmentation
- Disentangled Non-Local Neural Networks