A Generalizable Deep Learning System for Cardiac MRI

Abstract

Cardiac MRI allows for a comprehensive assessment of myocardial structure, function and tissue characteristics. Here we describe a foundational vision system for cardiac MRI, capable of representing the breadth of human cardiovascular disease and health. Our deep-learning model is trained via self-supervised contrastive learning, in which visual concepts in cine-sequence cardiac MRI scans are learned from the raw text of the accompanying radiology reports. We train and evaluate our model on data from four large academic clinical institutions in the United States. We additionally showcase the performance of our models on the UK BioBank and two additional publicly available external datasets. We explore emergent capabilities of our system and demonstrate remarkable performance across a range of tasks, including the problem of left-ventricular ejection fraction regression and the diagnosis of 39 different conditions such as cardiac amyloidosis and hypertrophic cardiomyopathy. We show that our deep-learning system is capable of not only contextualizing the staggering complexity of human cardiovascular disease but can be directed towards clinical problems of interest, yielding impressive, clinical-grade diagnostic accuracy with a fraction of the training data typically required for such tasks.

Published in Nature Biomedical Engineering; Supplementary Appendix available on publisher website. Code: https://github.com/rohanshad/cmr_transformer

References in corpus (10)

• Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing
• Shortcut Learning in Deep Neural Networks
• Predicting Cardiovascular Risk Factors from Retinal Fundus Photographs using Deep Learning
• Deep Learning-based Computational Pathology Predicts Origins for Cancers of Unknown Primary
• Predicting post-operative right ventricular failure using video-based deep learning
• Estimation of the volume of the left ventricle from MRI images using deep neural networks
• MedGemma Technical Report
• Merlin: A Computed Tomography Vision-Language Foundation Model and Dataset
• Towards Cardiac MRI Foundation Models: Comprehensive Visual-Tabular Representations for Whole-Heart Assessment and Beyond
• A versatile foundation model for cine cardiac magnetic resonance image analysis tasks

Cited by in corpus (1)

• Towards Cardiac MRI Foundation Models: Comprehensive Visual-Tabular Representations for Whole-Heart Assessment and Beyond