A Unified Efficient Pyramid Transformer for Semantic Segmentation
arXiv:2107.14209
Abstract
Semantic segmentation is a challenging problem due to difficulties in modeling context in complex scenes and class confusions along boundaries. Most literature either focuses on context modeling or boundary refinement, which is less generalizable in open-world scenarios. In this work, we advocate a unified framework(UN-EPT) to segment objects by considering both context information and boundary artifacts. We first adapt a sparse sampling strategy to incorporate the transformer-based attention mechanism for efficient context modeling. In addition, a separate spatial branch is introduced to capture image details for boundary refinement. The whole model can be trained in an end-to-end manner. We demonstrate promising performance on three popular benchmarks for semantic segmentation with low memory footprint. Code will be released soon.
References in corpus (13)
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- Rethinking Atrous Convolution for Semantic Image Segmentation
- Longformer: The Long-Document Transformer
- Multi-Scale Context Aggregation by Dilated Convolutions
- ParseNet: Looking Wider to See Better
- Richer Convolutional Features for Edge Detection
- DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
- OCNet: Object Context Network for Scene Parsing
- Generating Long Sequences with Sparse Transformers
- Reformer: The Efficient Transformer
- Devil in the Details: Towards Accurate Single and Multiple Human Parsing
- Improving Semantic Segmentation via Self-Training
- Class-wise Dynamic Graph Convolution for Semantic Segmentation