Enhancing Network Initialization for Medical AI Models Using Large-Scale, Unlabeled Natural Images

Abstract

Pre-training datasets, like ImageNet, have become the gold standard in medical image analysis. However, the emergence of self-supervised learning (SSL), which leverages unlabeled data to learn robust features, presents an opportunity to bypass the intensive labeling process. In this study, we explored if SSL for pre-training on non-medical images can be applied to chest radiographs and how it compares to supervised pre-training on non-medical images and on medical images. We utilized a vision transformer and initialized its weights based on (i) SSL pre-training on natural images (DINOv2), (ii) SL pre-training on natural images (ImageNet dataset), and (iii) SL pre-training on chest radiographs from the MIMIC-CXR database. We tested our approach on over 800,000 chest radiographs from six large global datasets, diagnosing more than 20 different imaging findings. Our SSL pre-training on curated images not only outperformed ImageNet-based pre-training (P<0.001 for all datasets) but, in certain cases, also exceeded SL on the MIMIC-CXR dataset. Our findings suggest that selecting the right pre-training strategy, especially with SSL, can be pivotal for improving artificial intelligence (AI)'s diagnostic accuracy in medical imaging. By demonstrating the promise of SSL in chest radiograph analysis, we underline a transformative shift towards more efficient and accurate AI models in medical imaging.

arXiv commentPublished in European Radiology Experimental

References in corpus (16)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Decoupled Weight Decay Regularization
• ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
• Self-supervised Learning: Generative or Contrastive
• DINOv2: Learning Robust Visual Features without Supervision
• PadChest: A large chest x-ray image dataset with multi-label annotated reports
• A review: Deep learning for medical image segmentation using multi-modality fusion
• Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty
• iBOT: Image BERT Pre-Training with Online Tokenizer
• NegBio: a high-performance tool for negation and uncertainty detection in radiology reports
• CheXtransfer: Performance and Parameter Efficiency of ImageNet Models for Chest X-Ray Interpretation
• Spreading vectors for similarity search
• Collaborative Training of Medical Artificial Intelligence Models with non-uniform Labels
• Weighted Ensemble Self-Supervised Learning
• A Self-Supervised Descriptor for Image Copy Detection

Cited by in corpus (3)

• Mind the Gap: Federated Learning Broadens Domain Generalization in Diagnostic AI Models
• Boosting multi-demographic federated learning for chest radiograph analysis using general-purpose self-supervised representations
• Resolution scaling governs DINOv3 transfer performance in chest radiograph classification