CheXtransfer: Performance and Parameter Efficiency of ImageNet Models for Chest X-Ray Interpretation

Abstract

Deep learning methods for chest X-ray interpretation typically rely on pretrained models developed for ImageNet. This paradigm assumes that better ImageNet architectures perform better on chest X-ray tasks and that ImageNet-pretrained weights provide a performance boost over random initialization. In this work, we compare the transfer performance and parameter efficiency of 16 popular convolutional architectures on a large chest X-ray dataset (CheXpert) to investigate these assumptions. First, we find no relationship between ImageNet performance and CheXpert performance for both models without pretraining and models with pretraining. Second, we find that, for models without pretraining, the choice of model family influences performance more than size within a family for medical imaging tasks. Third, we observe that ImageNet pretraining yields a statistically significant boost in performance across architectures, with a higher boost for smaller architectures. Fourth, we examine whether ImageNet architectures are unnecessarily large for CheXpert by truncating final blocks from pretrained models, and find that we can make models 3.25x more parameter-efficient on average without a statistically significant drop in performance. Our work contributes new experimental evidence about the relation of ImageNet to chest x-ray interpretation performance.

References in corpus (7)

• Distilling the Knowledge in a Neural Network
• Covid-19: Automatic detection from X-Ray images utilizing Transfer Learning with Convolutional Neural Networks
• A Survey of Model Compression and Acceleration for Deep Neural Networks
• Detecting Anemia from Retinal Fundus Images
• Comparing Different Deep Learning Architectures for Classification of Chest Radiographs
• MoCo-CXR: MoCo Pretraining Improves Representation and Transferability of Chest X-ray Models
• CheXternal: Generalization of Deep Learning Models for Chest X-ray Interpretation to Photos of Chest X-rays and External Clinical Settings

Cited by in corpus (9)

• Enhancing Network Initialization for Medical AI Models Using Large-Scale, Unlabeled Natural Images
• Effect of Pre-Training Scale on Intra- and Inter-Domain Full and Few-Shot Transfer Learning for Natural and Medical X-Ray Chest Images
• Significantly improving zero-shot X-ray pathology classification via fine-tuning pre-trained image-text encoders
• Multi-Dataset Multi-Task Learning for COVID-19 Prognosis
• Item Region-based Style Classification Network (IRSN): A Fashion Style Classifier Based on Domain Knowledge of Fashion Experts
• Benchmarking Foundation Models and Parameter-Efficient Fine-Tuning for Prognosis Prediction in Medical Imaging
• Learning to diagnose common thorax diseases on chest radiographs from radiology reports in Vietnamese
• Revisiting Hidden Representations in Transfer Learning for Medical Imaging
• Resolution scaling governs DINOv3 transfer performance in chest radiograph classification