Transformer Assisted Convolutional Network for Cell Instance Segmentation

Abstract

Region proposal based methods like R-CNN and Faster R-CNN models have proven to be extremely successful in object detection and segmentation tasks. Recently, Transformers have also gained popularity in the domain of Computer Vision, and are being utilised to improve the performance of conventional models. In this paper, we present a relatively new transformer based approach to enhance the performance of the conventional convolutional feature extractor in the existing region proposal based methods. Our approach merges the convolutional feature maps with transformer-based token embeddings by applying a projection operation similar to self-attention in transformers. The results of our experiments show that transformer assisted feature extractor achieves a significant improvement in mIoU (mean Intersection over Union) scores compared to vanilla convolutional backbone.

References in corpus (10)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• MMDetection: Open MMLab Detection Toolbox and Benchmark
• Semantic Instance Segmentation with a Discriminative Loss Function
• Visual Transformers: Token-based Image Representation and Processing for Computer Vision
• Semantic Instance Segmentation via Deep Metric Learning
• DetectoRS: Detecting Objects with Recursive Feature Pyramid and Switchable Atrous Convolution
• Adapting Mask-RCNN for Automatic Nucleus Segmentation
• Deep Watershed Transform for Instance Segmentation
• Distance to Center of Mass Encoding for Instance Segmentation