Graph-Segmenter: Graph Transformer with Boundary-aware Attention for Semantic Segmentation

Abstract

The transformer-based semantic segmentation approaches, which divide the image into different regions by sliding windows and model the relation inside each window, have achieved outstanding success. However, since the relation modeling between windows was not the primary emphasis of previous work, it was not fully utilized. To address this issue, we propose a Graph-Segmenter, including a Graph Transformer and a Boundary-aware Attention module, which is an effective network for simultaneously modeling the more profound relation between windows in a global view and various pixels inside each window as a local one, and for substantial low-cost boundary adjustment. Specifically, we treat every window and pixel inside the window as nodes to construct graphs for both views and devise the Graph Transformer. The introduced boundary-aware attention module optimizes the edge information of the target objects by modeling the relationship between the pixel on the object's edge. Extensive experiments on three widely used semantic segmentation datasets (Cityscapes, ADE-20k and PASCAL Context) demonstrate that our proposed network, a Graph Transformer with Boundary-aware Attention, can achieve state-of-the-art segmentation performance.

References in corpus (13)

• Neural Machine Translation by Jointly Learning to Align and Translate
• A Comprehensive Survey on Graph Neural Networks
• Fully Convolutional Networks for Semantic Segmentation
• Spectral Networks and Locally Connected Networks on Graphs
• Deformable DETR: Deformable Transformers for End-to-End Object Detection
• A Survey of Deep Learning Techniques for Autonomous Driving
• Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges
• Twins: Revisiting the Design of Spatial Attention in Vision Transformers
• OCNet: Object Context Network for Scene Parsing
• Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation
• Searching for Efficient Multi-Scale Architectures for Dense Image Prediction
• Dual Graph Convolutional Network for Semantic Segmentation
• Graph Convolution Machine for Context-aware Recommender System

Cited by in corpus (2)

• PolyFormer: Scalable Node-wise Filters via Polynomial Graph Transformer
• Deep Common Feature Mining for Efficient Video Semantic Segmentation