LT-ViT: A Vision Transformer for multi-label Chest X-ray classification

Abstract

Vision Transformers (ViTs) are widely adopted in medical imaging tasks, and some existing efforts have been directed towards vision-language training for Chest X-rays (CXRs). However, we envision that there still exists a potential for improvement in vision-only training for CXRs using ViTs, by aggregating information from multiple scales, which has been proven beneficial for non-transformer networks. Hence, we have developed LT-ViT, a transformer that utilizes combined attention between image tokens and randomly initialized auxiliary tokens that represent labels. Our experiments demonstrate that LT-ViT (1) surpasses the state-of-the-art performance using pure ViTs on two publicly available CXR datasets, (2) is generalizable to other pre-training methods and therefore is agnostic to model initialization, and (3) enables model interpretability without grad-cam and its variants.

arXiv comment5 pages, 2 figures

References in corpus (6)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases
• MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs
• Query2Label: A Simple Transformer Way to Multi-Label Classification
• Multi-Granularity Cross-modal Alignment for Generalized Medical Visual Representation Learning
• SPCXR: Self-supervised Pretraining using Chest X-rays Towards a Domain Specific Foundation Model

Cited by in corpus (2)

• ASiT: Local-Global Audio Spectrogram vIsion Transformer for Event Classification
• Mask of truth: model sensitivity to unexpected regions of medical images