Is attention all you need in medical image analysis? A review
arXiv:2307.12775 · doi:10.1109/JBHI.2023.3348436
Abstract
Medical imaging is a key component in clinical diagnosis, treatment planning and clinical trial design, accounting for almost 90% of all healthcare data. CNNs achieved performance gains in medical image analysis (MIA) over the last years. CNNs can efficiently model local pixel interactions and be trained on small-scale MI data. The main disadvantage of typical CNN models is that they ignore global pixel relationships within images, which limits their generalisation ability to understand out-of-distribution data with different 'global' information. The recent progress of Artificial Intelligence gave rise to Transformers, which can learn global relationships from data. However, full Transformer models need to be trained on large-scale data and involve tremendous computational complexity. Attention and Transformer compartments (Transf/Attention) which can well maintain properties for modelling global relationships, have been proposed as lighter alternatives of full Transformers. Recently, there is an increasing trend to co-pollinate complementary local-global properties from CNN and Transf/Attention architectures, which led to a new era of hybrid models. The past years have witnessed substantial growth in hybrid CNN-Transf/Attention models across diverse MIA problems. In this systematic review, we survey existing hybrid CNN-Transf/Attention models, review and unravel key architectural designs, analyse breakthroughs, and evaluate current and future opportunities as well as challenges. We also introduced a comprehensive analysis framework on generalisation opportunities of scientific and clinical impact, based on which new data-driven domain generalisation and adaptation methods can be stimulated.
References in corpus (14)
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises
- Recent advances and clinical applications of deep learning in medical image analysis
- ResViT: Residual vision transformers for multi-modal medical image synthesis
- An Explainable 3D Residual Self-Attention Deep Neural Network FOR Joint Atrophy Localization and Alzheimer's Disease Diagnosis using Structural MRI
- Generalized Radiograph Representation Learning via Cross-supervision between Images and Free-text Radiology Reports
- Self-Attentive Spatial Adaptive Normalization for Cross-Modality Domain Adaptation
- Relational Modeling for Robust and Efficient Pulmonary Lobe Segmentation in CT Scans
- Noise Conscious Training of Non Local Neural Network powered by Self Attentive Spectral Normalized Markovian Patch GAN for Low Dose CT Denoising
- ProCAN: Progressive Growing Channel Attentive Non-Local Network for Lung Nodule Classification
- ChatCAD: Interactive Computer-Aided Diagnosis on Medical Image using Large Language Models
- Robust End-to-End Focal Liver Lesion Detection using Unregistered Multiphase Computed Tomography Images
- Weakly Supervised Attention Model for RV StrainClassification from volumetric CTPA Scans
- Automated Olfactory Bulb Segmentation on High Resolutional T2-Weighted MRI