CA-Net: Comprehensive Attention Convolutional Neural Networks for Explainable Medical Image Segmentation

Abstract

Accurate medical image segmentation is essential for diagnosis and treatment planning of diseases. Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they are still challenged by complicated conditions where the segmentation target has large variations of position, shape and scale, and existing CNNs have a poor explainability that limits their application to clinical decisions. In this work, we make extensive use of multiple attentions in a CNN architecture and propose a comprehensive attention-based CNN (CA-Net) for more accurate and explainable medical image segmentation that is aware of the most important spatial positions, channels and scales at the same time. In particular, we first propose a joint spatial attention module to make the network focus more on the foreground region. Then, a novel channel attention module is proposed to adaptively recalibrate channel-wise feature responses and highlight the most relevant feature channels. Also, we propose a scale attention module implicitly emphasizing the most salient feature maps among multiple scales so that the CNN is adaptive to the size of an object. Extensive experiments on skin lesion segmentation from ISIC 2018 and multi-class segmentation of fetal MRI found that our proposed CA-Net significantly improved the average segmentation Dice score from 87.77% to 92.08% for skin lesion, 84.79% to 87.08% for the placenta and 93.20% to 95.88% for the fetal brain respectively compared with U-Net. It reduced the model size to around 15 times smaller with close or even better accuracy compared with state-of-the-art DeepLabv3+. In addition, it has a much higher explainability than existing networks by visualizing the attention weight maps. Our code is available at https://github.com/HiLab-git/CA-Net

References in corpus (8)

• A Survey on Deep Learning in Medical Image Analysis
• Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation
• Striving for Simplicity: The All Convolutional Net
• Interactive Medical Image Segmentation using Deep Learning with Image-specific Fine-tuning
• DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation
• On the Compactness, Efficiency, and Representation of 3D Convolutional Networks: Brain Parcellation as a Pretext Task
• SLSDeep: Skin Lesion Segmentation Based on Dilated Residual and Pyramid Pooling Networks
• Real-Time Automatic Fetal Brain Extraction in Fetal MRI by Deep Learning

Cited by in corpus (13)

• A Survey on Deep Learning for Skin Lesion Segmentation
• Explainable, Domain-Adaptive, and Federated Artificial Intelligence in Medicine
• PyMIC: A deep learning toolkit for annotation-efficient medical image segmentation
• HiDiff: Hybrid Diffusion Framework for Medical Image Segmentation
• Explainable Artificial Intelligence to Detect Image Spam Using Convolutional Neural Network
• MLN-net: A multi-source medical image segmentation method for clustered microcalcifications using multiple layer normalization
• Investigating Shift-Variance of Convolutional Neural Networks in Ultrasound Image Segmentation
• GLIMS: Attention-Guided Lightweight Multi-Scale Hybrid Network for Volumetric Semantic Segmentation
• HST-MRF: Heterogeneous Swin Transformer with Multi-Receptive Field for Medical Image Segmentation
• Decoupled Pyramid Correlation Network for Liver Tumor Segmentation from CT images
• FAS-UNet: A Novel FAS-driven Unet to Learn Variational Image Segmentation
• MS-DCANet: A Novel Segmentation Network For Multi-Modality COVID-19 Medical Images
• Multi-Scale Grouped Prototypes for Interpretable Semantic Segmentation