Vision Transformer using Low-level Chest X-ray Feature Corpus for COVID-19 Diagnosis and Severity Quantification

Abstract

Developing a robust algorithm to diagnose and quantify the severity of COVID-19 using Chest X-ray (CXR) requires a large number of well-curated COVID-19 datasets, which is difficult to collect under the global COVID-19 pandemic. On the other hand, CXR data with other findings are abundant. This situation is ideally suited for the Vision Transformer (ViT) architecture, where a lot of unlabeled data can be used through structural modeling by the self-attention mechanism. However, the use of existing ViT is not optimal, since feature embedding through direct patch flattening or ResNet backbone in the standard ViT is not intended for CXR. To address this problem, here we propose a novel Vision Transformer that utilizes low-level CXR feature corpus obtained from a backbone network that extracts common CXR findings. Specifically, the backbone network is first trained with large public datasets to detect common abnormal findings such as consolidation, opacity, edema, etc. Then, the embedded features from the backbone network are used as corpora for a Transformer model for the diagnosis and the severity quantification of COVID-19. We evaluate our model on various external test datasets from totally different institutions to evaluate the generalization capability. The experimental results confirm that our model can achieve the state-of-the-art performance in both diagnosis and severity quantification tasks with superior generalization capability, which are sine qua non of widespread deployment.

arXiv comment13 pages

References in corpus (10)

• Transformers in Vision: A Survey
• Covid-19: Automatic detection from X-Ray images utilizing Transfer Learning with Convolutional Neural Networks
• Deformable DETR: Deformable Transformers for End-to-End Object Detection
• Confounding variables can degrade generalization performance of radiological deep learning models
• COVIDX-Net: A Framework of Deep Learning Classifiers to Diagnose COVID-19 in X-Ray Images
• SmoothGrad: removing noise by adding noise
• Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers
• The challenges of deploying artificial intelligence models in a rapidly evolving pandemic
• Full-Gradient Representation for Neural Network Visualization
• Weakly Supervised Lesion Localization With Probabilistic-CAM Pooling

Cited by in corpus (1)

• Federated Split Vision Transformer for COVID-19 CXR Diagnosis using Task-Agnostic Training