Video Polyp Segmentation: A Deep Learning Perspective

Abstract

We present the first comprehensive video polyp segmentation (VPS) study in the deep learning era. Over the years, developments in VPS are not moving forward with ease due to the lack of large-scale fine-grained segmentation annotations. To address this issue, we first introduce a high-quality frame-by-frame annotated VPS dataset, named SUN-SEG, which contains 158,690 colonoscopy frames from the well-known SUN-database. We provide additional annotations with diverse types, i.e., attribute, object mask, boundary, scribble, and polygon. Second, we design a simple but efficient baseline, dubbed PNS+, consisting of a global encoder, a local encoder, and normalized self-attention (NS) blocks. The global and local encoders receive an anchor frame and multiple successive frames to extract long-term and short-term spatial-temporal representations, which are then progressively updated by two NS blocks. Extensive experiments show that PNS+ achieves the best performance and real-time inference speed (170fps), making it a promising solution for the VPS task. Third, we extensively evaluate 13 representative polyp/object segmentation models on our SUN-SEG dataset and provide attribute-based comparisons. Finally, we discuss several open issues and suggest possible research directions for the VPS community.

Accepted by Machine Intelligence Research 2022 (Project Page: https://github.com/GewelsJI/VPS)

References in corpus (3)

• Real-Time Polyp Detection, Localization and Segmentation in Colonoscopy Using Deep Learning
• UACANet: Uncertainty Augmented Context Attention for Polyp Segmentation
• Deep Gradient Learning for Efficient Camouflaged Object Detection

Cited by in corpus (11)

• Deep Gradient Learning for Efficient Camouflaged Object Detection
• Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-world Applications
• ZoomNeXt: A Unified Collaborative Pyramid Network for Camouflaged Object Detection
• A Survey on Deep Learning for Polyp Segmentation: Techniques, Challenges and Future Trends
• QMaxViT-Unet+: A Query-Based MaxViT-Unet with Edge Enhancement for Scribble-Supervised Segmentation of Medical Images
• How Good is Google Bard's Visual Understanding? An Empirical Study on Open Challenges
• Rethinking Polyp Segmentation from an Out-of-Distribution Perspective
• Acquiring Weak Annotations for Tumor Localization in Temporal and Volumetric Data
• Frontiers in Intelligent Colonoscopy
• Temporal Transductive Inference for Few-Shot Video Object Segmentation
• DyABD: The Abdominal Muscle Segmentation in Dynamic MRI Benchmark