Dual Cross-Attention for Medical Image Segmentation

Abstract

We propose Dual Cross-Attention (DCA), a simple yet effective attention module that is able to enhance skip-connections in U-Net-based architectures for medical image segmentation. DCA addresses the semantic gap between encoder and decoder features by sequentially capturing channel and spatial dependencies across multi-scale encoder features. First, the Channel Cross-Attention (CCA) extracts global channel-wise dependencies by utilizing cross-attention across channel tokens of multi-scale encoder features. Then, the Spatial Cross-Attention (SCA) module performs cross-attention to capture spatial dependencies across spatial tokens. Finally, these fine-grained encoder features are up-sampled and connected to their corresponding decoder parts to form the skip-connection scheme. Our proposed DCA module can be integrated into any encoder-decoder architecture with skip-connections such as U-Net and its variants. We test our DCA module by integrating it into six U-Net-based architectures such as U-Net, V-Net, R2Unet, ResUnet++, DoubleUnet and MultiResUnet. Our DCA module shows Dice Score improvements up to 2.05% on GlaS, 2.74% on MoNuSeg, 1.37% on CVC-ClinicDB, 1.12% on Kvasir-Seg and 1.44% on Synapse datasets. Our codes are available at: https://github.com/gorkemcanates/Dual-Cross-Attention

arXiv commentCode: https://github.com/gorkemcanates/Dual-Cross-Attention

References in corpus (10)

• Adam: A Method for Stochastic Optimization
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation
• Gaussian Error Linear Units (GELUs)
• Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations
• MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation
• nnFormer: Interleaved Transformer for Volumetric Segmentation
• MISSFormer: An Effective Medical Image Segmentation Transformer
• LeViT-UNet: Make Faster Encoders with Transformer for Medical Image Segmentation
• Focal-UNet: UNet-like Focal Modulation for Medical Image Segmentation

Cited by in corpus (2)

• CFFormer: Cross CNN-Transformer Channel Attention and Spatial Feature Fusion for Improved Segmentation of Heterogeneous Medical Images
• Deep Learning-based Bathymetry Retrieval without In-situ Depths using Remote Sensing Imagery and SfM-MVS DSMs with Data Gaps