DeepSMILE: Contrastive self-supervised pre-training benefits MSI and HRD classification directly from H&E whole-slide images in colorectal and breast cancer

Abstract

We propose a Deep learning-based weak label learning method for analyzing whole slide images (WSIs) of Hematoxylin and Eosin (H&E) stained tumor tissue not requiring pixel-level or tile-level annotations using Self-supervised pre-training and heterogeneity-aware deep Multiple Instance LEarning (DeepSMILE). We apply DeepSMILE to the task of Homologous recombination deficiency (HRD) and microsatellite instability (MSI) prediction. We utilize contrastive self-supervised learning to pre-train a feature extractor on histopathology tiles of cancer tissue. Additionally, we use variability-aware deep multiple instance learning to learn the tile feature aggregation function while modeling tumor heterogeneity. For MSI prediction in a tumor-annotated and color normalized subset of TCGA-CRC (n=360 patients), contrastive self-supervised learning improves the tile supervision baseline from 0.77 to 0.87 AUROC, on par with our proposed DeepSMILE method. On TCGA-BC (n=1041 patients) without any manual annotations, DeepSMILE improves HRD classification performance from 0.77 to 0.81 AUROC compared to tile supervision with either a self-supervised or ImageNet pre-trained feature extractor. Our proposed methods reach the baseline performance using only 40% of the labeled data on both datasets. These improvements suggest we can use standard self-supervised learning techniques combined with multiple instance learning in the histopathology domain to improve genomic label classification performance with fewer labeled data.

arXiv commentMain paper: 14 pages, 2 tables, 1 algorithm, 3 figures. Supplementary material: 3 pages

References in corpus (12)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• A Simple Framework for Contrastive Learning of Visual Representations
• Representation Learning with Contrastive Predictive Coding
• Bootstrap your own latent: A new approach to self-supervised Learning
• Improved Baselines with Momentum Contrastive Learning
• Attention-based Deep Multiple Instance Learning
• Deep Learning-based Computational Pathology Predicts Origins for Cancers of Unknown Primary
• Transfusion: Understanding Transfer Learning for Medical Imaging
• Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty
• Neural Image Compression for Gigapixel Histopathology Image Analysis
• Streaming convolutional neural networks for end-to-end learning with multi-megapixel images
• Extending Unsupervised Neural Image Compression With Supervised Multitask Learning

Cited by in corpus (3)

• Predicting microsatellite instability and key biomarkers in colorectal cancer from H&E-stained images: Achieving SOTA predictive performance with fewer data using Swin Transformer
• Exploring the Interplay Between Colorectal Cancer Subtypes Genomic Variants and Cellular Morphology: A Deep-Learning Approach
• Explainable AI for computational pathology identifies model limitations and tissue biomarkers