MISSFormer: An Effective Medical Image Segmentation Transformer

Abstract

The CNN-based methods have achieved impressive results in medical image segmentation, but they failed to capture the long-range dependencies due to the inherent locality of the convolution operation. Transformer-based methods are recently popular in vision tasks because of their capacity for long-range dependencies and promising performance. However, it lacks in modeling local context. In this paper, taking medical image segmentation as an example, we present MISSFormer, an effective and powerful Medical Image Segmentation tranSFormer. MISSFormer is a hierarchical encoder-decoder network with two appealing designs: 1) A feed-forward network is redesigned with the proposed Enhanced Transformer Block, which enhances the long-range dependencies and supplements the local context, making the feature more discriminative. 2) We proposed Enhanced Transformer Context Bridge, different from previous methods of modeling only global information, the proposed context bridge with the enhanced transformer block extracts the long-range dependencies and local context of multi-scale features generated by our hierarchical transformer encoder. Driven by these two designs, the MISSFormer shows a solid capacity to capture more discriminative dependencies and context in medical image segmentation. The experiments on multi-organ and cardiac segmentation tasks demonstrate the superiority, effectiveness and robustness of our MISSFormer, the experimental results of MISSFormer trained from scratch even outperform state-of-the-art methods pre-trained on ImageNet. The core designs can be generalized to other visual segmentation tasks. The code has been released on Github: https://github.com/ZhifangDeng/MISSFormer

10 pages, 6 figures

References in corpus (11)

• TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation
• PVT v2: Improved Baselines with Pyramid Vision Transformer
• Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation
• SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
• Conditional Positional Encodings for Vision Transformers
• LocalViT: Analyzing Locality in Vision Transformers
• UNETR: Transformers for 3D Medical Image Segmentation
• TransBTS: Multimodal Brain Tumor Segmentation Using Transformer
• Uformer: A General U-Shaped Transformer for Image Restoration
• LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference
• CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation

Cited by in corpus (7)

• Dual Cross-Attention for Medical Image Segmentation
• Transformers in Healthcare: A Survey
• HistoSeg : Quick attention with multi-loss function for multi-structure segmentation in digital histology images
• Optimizing Vision Transformers for Medical Image Segmentation
• An Efficient Dual-Line Decoder Network with Multi-Scale Convolutional Attention for Multi-organ Segmentation
• Leveraging Self-Supervised Vision Transformers for Segmentation-based Transfer Function Design
• Improving FHB Screening in Wheat Breeding Using an Efficient Transformer Model