Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformers

Abstract

Most polyp segmentation methods use CNNs as their backbone, leading to two key issues when exchanging information between the encoder and decoder: 1) taking into account the differences in contribution between different-level features and 2) designing an effective mechanism for fusing these features. Unlike existing CNN-based methods, we adopt a transformer encoder, which learns more powerful and robust representations. In addition, considering the image acquisition influence and elusive properties of polyps, we introduce three standard modules, including a cascaded fusion module (CFM), a camouflage identification module (CIM), and a similarity aggregation module (SAM). Among these, the CFM is used to collect the semantic and location information of polyps from high-level features; the CIM is applied to capture polyp information disguised in low-level features, and the SAM extends the pixel features of the polyp area with high-level semantic position information to the entire polyp area, thereby effectively fusing cross-level features. The proposed model, named Polyp-PVT, effectively suppresses noises in the features and significantly improves their expressive capabilities. Extensive experiments on five widely adopted datasets show that the proposed model is more robust to various challenging situations (e.g., appearance changes, small objects, rotation) than existing representative methods. The proposed model is available at https://github.com/DengPingFan/Polyp-PVT.

Accepted to CAAI AIR 2023

References in corpus (11)

• Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
• PVT v2: Improved Baselines with Pyramid Vision Transformer
• Concealed Object Detection
• Real-Time Polyp Detection, Localization and Segmentation in Colonoscopy Using Deep Learning
• Automated polyp detection in colon capsule endoscopy
• UACANet: Uncertainty Augmented Context Attention for Polyp Segmentation
• Video Polyp Segmentation: A Deep Learning Perspective
• Progressively Normalized Self-Attention Network for Video Polyp Segmentation
• Polyp Detection and Segmentation using Mask R-CNN: Does a Deeper Feature Extractor CNN Always Perform Better?
• Towards a Computed-Aided Diagnosis System in Colonoscopy: Automatic Polyp Segmentation Using Convolution Neural Networks
• Superpixel Based Segmentation and Classification of Polyps in Wireless Capsule Endoscopy

Cited by in corpus (9)

• HiDiff: Hybrid Diffusion Framework for Medical Image Segmentation
• SliceMamba with Neural Architecture Search for Medical Image Segmentation
• DPE-Net: Dual-Parallel Encoder Based Network for Semantic Segmentation of Polyps
• Advances in Medical Image Segmentation: A Comprehensive Survey with a Focus on Lumbar Spine Applications
• Deep Bayesian segmentation for colon polyps: Well-calibrated predictions in medical imaging
• An Efficient Dual-Line Decoder Network with Multi-Scale Convolutional Attention for Multi-organ Segmentation
• BiSeg-SAM: Weakly-Supervised Post-Processing Framework for Boosting Binary Segmentation in Segment Anything Models
• RTA-Former: Reverse Transformer Attention for Polyp Segmentation
• Soft Dice Confidence: A Near-Optimal Confidence Estimator for Selective Prediction in Semantic Segmentation