Segmenting Transparent Object in the Wild with Transformer

Abstract

This work presents a new fine-grained transparent object segmentation dataset, termed Trans10K-v2, extending Trans10K-v1, the first large-scale transparent object segmentation dataset. Unlike Trans10K-v1 that only has two limited categories, our new dataset has several appealing benefits. (1) It has 11 fine-grained categories of transparent objects, commonly occurring in the human domestic environment, making it more practical for real-world application. (2) Trans10K-v2 brings more challenges for the current advanced segmentation methods than its former version. Furthermore, a novel transformer-based segmentation pipeline termed Trans2Seg is proposed. Firstly, the transformer encoder of Trans2Seg provides the global receptive field in contrast to CNN's local receptive field, which shows excellent advantages over pure CNN architectures. Secondly, by formulating semantic segmentation as a problem of dictionary look-up, we design a set of learnable prototypes as the query of Trans2Seg's transformer decoder, where each prototype learns the statistics of one category in the whole dataset. We benchmark more than 20 recent semantic segmentation methods, demonstrating that Trans2Seg significantly outperforms all the CNN-based methods, showing the proposed algorithm's potential ability to solve transparent object segmentation.

Tech. Report

References in corpus (13)

• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Conditional Random Fields as Recurrent Neural Networks
• Deformable DETR: Deformable Transformers for End-to-End Object Detection
• DUNet: A deformable network for retinal vessel segmentation
• OCNet: Object Context Network for Scene Parsing
• Fast-SCNN: Fast Semantic Segmentation Network
• TransTrack: Multiple Object Tracking with Transformer
• ContextNet: Exploring Context and Detail for Semantic Segmentation in Real-time
• DABNet: Depth-wise Asymmetric Bottleneck for Real-time Semantic Segmentation
• Pre-Trained Image Processing Transformer
• End-to-End Video Instance Segmentation with Transformers
• Feature Pyramid Encoding Network for Real-time Semantic Segmentation
• Segmenting Transparent Objects in the Wild

Cited by in corpus (4)

• SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
• Glass Segmentation with RGB-Thermal Image Pairs
• Fully Transformer Networks for Semantic Image Segmentation
• Salient Mask-Guided Vision Transformer for Fine-Grained Classification