Predicting Cardiovascular Risk Factors from Retinal Fundus Photographs using Deep Learning

Abstract

Traditionally, medical discoveries are made by observing associations and then designing experiments to test these hypotheses. However, observing and quantifying associations in images can be difficult because of the wide variety of features, patterns, colors, values, shapes in real data. In this paper, we use deep learning, a machine learning technique that learns its own features, to discover new knowledge from retinal fundus images. Using models trained on data from 284,335 patients, and validated on two independent datasets of 12,026 and 999 patients, we predict cardiovascular risk factors not previously thought to be present or quantifiable in retinal images, such as such as age (within 3.26 years), gender (0.97 AUC), smoking status (0.71 AUC), HbA1c (within 1.39%), systolic blood pressure (within 11.23mmHg) as well as major adverse cardiac events (0.70 AUC). We further show that our models used distinct aspects of the anatomy to generate each prediction, such as the optic disc or blood vessels, opening avenues of further research.

References in corpus (5)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
• Grader variability and the importance of reference standards for evaluating machine learning models for diabetic retinopathy
• Describing Multimedia Content using Attention-based Encoder--Decoder Networks

Cited by in corpus (76)

• On the Opportunities and Risks of Foundation Models
• An overview of deep learning in medical imaging focusing on MRI
• The Medical Segmentation Decathlon
• REFUGE Challenge: A Unified Framework for Evaluating Automated Methods for Glaucoma Assessment from Fundus Photographs
• Deep Learning-based Computational Pathology Predicts Origins for Cancers of Unknown Primary
• Reading Race: AI Recognises Patient's Racial Identity In Medical Images
• DeepSeeNet: A deep learning model for automated classification of patient-based age-related macular degeneration severity from color fundus photographs
• A Review of Challenges and Opportunities in Machine Learning for Health
• High-Throughput Precision Phenotyping of Left Ventricular Hypertrophy with Cardiovascular Deep Learning
• Detecting Anemia from Retinal Fundus Images
• Contactless Electrocardiogram Monitoring with Millimeter Wave Radar
• Differential Privacy-enabled Federated Learning for Sensitive Health Data
• Exploiting Epistemic Uncertainty of Anatomy Segmentation for Anomaly Detection in Retinal OCT
• Predicting optical coherence tomography-derived diabetic macular edema grades from fundus photographs using deep learning
• Deep learning on fundus images detects glaucoma beyond the optic disc
• Detecting Shortcut Learning for Fair Medical AI using Shortcut Testing
• Detecting hidden signs of diabetes in external eye photographs
• VTGAN: Semi-supervised Retinal Image Synthesis and Disease Prediction using Vision Transformers
• VisionFM: a Multi-Modal Multi-Task Vision Foundation Model for Generalist Ophthalmic Artificial Intelligence
• Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction
• A Survey of Deep Learning for Scientific Discovery
• Using generative AI to investigate medical imagery models and datasets
• Medical Imaging and Machine Learning
• A Novel Deep Learning Pipeline for Retinal Vessel Detection in Fluorescein Angiography
• COVIDx-US -- An open-access benchmark dataset of ultrasound imaging data for AI-driven COVID-19 analytics
• Hi Sigma, do I have the Coronavirus?: Call for a New Artificial Intelligence Approach to Support Health Care Professionals Dealing With The COVID-19 Pandemic
• Deep Learning Discovery of Demographic Biomarkers in Echocardiography
• Embedded deep learning in ophthalmology: Making ophthalmic imaging smarter
• WarpPINN: Cine-MR image registration with physics-informed neural networks
• Artificial Intelligence and Diabetes Mellitus: An Inside Look Through the Retina
• Rice Classification Using Spatio-Spectral Deep Convolutional Neural Network
• Attention2AngioGAN: Synthesizing Fluorescein Angiography from Retinal Fundus Images using Generative Adversarial Networks
• Fundus2Angio: A Conditional GAN Architecture for Generating Fluorescein Angiography Images from Retinal Fundus Photography
• Recognition in Terra Incognita
• Interpretable deep learning for nuclear deformation in heavy ion collisions
• Artificial Intelligence in Assessing Cardiovascular Diseases and Risk Factors via Retinal Fundus Images: A Review of the Last Decade
• Attention Based Glaucoma Detection: A Large-scale Database and CNN Model
• Audit to Forget: A Unified Method to Revoke Patients' Private Data in Intelligent Healthcare
• Representation Learning for Electronic Health Records
• Integrating electrocardiogram and fundus images for early detection of cardiovascular diseases
• m2caiSeg: Semantic Segmentation of Laparoscopic Images using Convolutional Neural Networks
• BEHRT: Transformer for Electronic Health Records
• CaRENets: Compact and Resource-Efficient CNN for Homomorphic Inference on Encrypted Medical Images
• A Benchmark of Ocular Disease Intelligent Recognition: One Shot for Multi-disease Detection
• On the Semantic Interpretability of Artificial Intelligence Models
• AI Fairness via Domain Adaptation
• Beyond Low Earth Orbit: Biomonitoring, Artificial Intelligence, and Precision Space Health
• Recent Advances in Medical Image Classification
• Predicting risk of late age-related macular degeneration using deep learning
• A Generalizable Deep Learning System for Cardiac MRI
• Camera Adaptation for Fundus-Image-Based CVD Risk Estimation
• The Eye as a Window to Systemic Health: A Survey of Retinal Imaging from Classical Techniques to Oculomics
• Spiking Neural Predictive Coding for Continual Learning from Data Streams
• It's easy to fool yourself: Case studies on identifying bias and confounding in bio-medical datasets
• Unsupervised deep learning for grading of age-related macular degeneration using retinal fundus images
• Integrating Deep Learning with Fundus and Optical Coherence Tomography for Cardiovascular Disease Prediction
• Disentangling representations of retinal images with generative models
• Applications of Deep Learning in Fundus Images: A Review
• Assessment of central serous chorioretinopathy (CSC) depicted on color fundus photographs using deep Learning
• Efficient Supervision for Robot Learning via Imitation, Simulation, and Adaptation
• Predicting the risk of pancreatic cancer with a CT-based ensemble AI algorithm
• Unsupervised Learning of Audio Perception for Robotics Applications: Learning to Project Data to T-SNE/UMAP space
• Algorithm Fairness in AI for Medicine and Healthcare
• Detecting cutaneous basal cell carcinomas in ultra-high resolution and weakly labelled histopathological images
• The Roles of Supervised Machine Learning in Systems Neuroscience
• Identifying Pediatric Vascular Anomalies With Deep Learning
• Artificial Intelligence for Pediatric Ophthalmology
• Using a Cross-Task Grid of Linear Probes to Interpret CNN Model Predictions On Retinal Images
• On the Robustness of Pretraining and Self-Supervision for a Deep Learning-based Analysis of Diabetic Retinopathy
• SoQal: Selective Oracle Questioning in Active Learning
• IIT Kanpur Consulting Group: Using Machine Learning and Management Consulting for Social Good
• Differentially private training of neural networks with Langevin dynamics for calibrated predictive uncertainty
• How Much Does It Hurt: A Deep Learning Framework for Chronic Pain Score Assessment
• Voting of predictive models for clinical outcomes: consensus of algorithms for the early prediction of sepsis from clinical data and an analysis of the PhysioNet/Computing in Cardiology Challenge 2019
• Cognitive computation of brain disorders based primarily on ocular responses
• Deep learning-based quality filtering of mechanically exfoliated 2D crystals