BLADE: Box-Level Supervised Amodal Segmentation through Directed Expansion

Abstract

Perceiving the complete shape of occluded objects is essential for human and machine intelligence. While the amodal segmentation task is to predict the complete mask of partially occluded objects, it is time-consuming and labor-intensive to annotate the pixel-level ground truth amodal masks. Box-level supervised amodal segmentation addresses this challenge by relying solely on ground truth bounding boxes and instance classes as supervision, thereby alleviating the need for exhaustive pixel-level annotations. Nevertheless, current box-level methodologies encounter limitations in generating low-resolution masks and imprecise boundaries, failing to meet the demands of practical real-world applications. We present a novel solution to tackle this problem by introducing a directed expansion approach from visible masks to corresponding amodal masks. Our approach involves a hybrid end-to-end network based on the overlapping region - the area where different instances intersect. Diverse segmentation strategies are applied for overlapping regions and non-overlapping regions according to distinct characteristics. To guide the expansion of visible masks, we introduce an elaborately-designed connectivity loss for overlapping regions, which leverages correlations with visible masks and facilitates accurate amodal segmentation. Experiments are conducted on several challenging datasets and the results show that our proposed method can outperform existing state-of-the-art methods with large margins.

arXiv commentAccepted to AAAI 2024

References in corpus (12)

• FCOS: Fully Convolutional One-Stage Object Detection
• Dynamic Filter Networks
• Simple Does It: Weakly Supervised Instance and Semantic Segmentation
• BoxInst: High-Performance Instance Segmentation with Box Annotations
• Image Amodal Completion: A Survey
• AISFormer: Amodal Instance Segmentation with Transformer
• Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers
• Robust Object Detection under Occlusion with Context-Aware CompositionalNets
• Amodal Segmentation Based on Visible Region Segmentation and Shape Prior
• Amodal Segmentation through Out-of-Task and Out-of-Distribution Generalization with a Bayesian Model
• Amodal Cityscapes: A New Dataset, its Generation, and an Amodal Semantic Segmentation Challenge Baseline
• PanoDR: Spherical Panorama Diminished Reality for Indoor Scenes