Self-Supervision Closes the Gap Between Weak and Strong Supervision in Histology

Abstract

One of the biggest challenges for applying machine learning to histopathology is weak supervision: whole-slide images have billions of pixels yet often only one global label. The state of the art therefore relies on strongly-supervised model training using additional local annotations from domain experts. However, in the absence of detailed annotations, most weakly-supervised approaches depend on a frozen feature extractor pre-trained on ImageNet. We identify this as a key weakness and propose to train an in-domain feature extractor on histology images using MoCo v2, a recent self-supervised learning algorithm. Experimental results on Camelyon16 and TCGA show that the proposed extractor greatly outperforms its ImageNet counterpart. In particular, our results improve the weakly-supervised state of the art on Camelyon16 from 91.4% to 98.7% AUC, thereby closing the gap with strongly-supervised models that reach 99.3% AUC. Through these experiments, we demonstrate that feature extractors trained via self-supervised learning can act as drop-in replacements to significantly improve existing machine learning techniques in histology. Lastly, we show that the learned embedding space exhibits biologically meaningful separation of tissue structures.

arXiv commentAccepted as a poster for the ML4H 2020 NeurIPS workshop

References in corpus (14)

• A Simple Framework for Contrastive Learning of Visual Representations
• Deep Residual Learning for Image Recognition
• Bootstrap your own latent: A new approach to self-supervised Learning
• Improved Baselines with Momentum Contrastive Learning
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
• Momentum Contrast for Unsupervised Visual Representation Learning
• Deep Learning for Identifying Metastatic Breast Cancer
• Automated Gleason Grading of Prostate Biopsies using Deep Learning
• Whole Slide Images based Cancer Survival Prediction using Attention Guided Deep Multiple Instance Learning Networks
• Big Self-Supervised Models are Strong Semi-Supervised Learners
• Neural Image Compression for Gigapixel Histopathology Image Analysis
• Streaming convolutional neural networks for end-to-end learning with multi-megapixel images
• Classification and Disease Localization in Histopathology Using Only Global Labels: A Weakly-Supervised Approach
• Semi-Supervised Histology Classification using Deep Multiple Instance Learning and Contrastive Predictive Coding

Cited by in corpus (4)

• Towards Label-efficient Automatic Diagnosis and Analysis: A Comprehensive Survey of Advanced Deep Learning-based Weakly-supervised, Semi-supervised and Self-supervised Techniques in Histopathological Image Analysis
• Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning
• Self supervised learning improves dMMR/MSI detection from histology slides across multiple cancers
• Towards Fine-grained Visual Representations by Combining Contrastive Learning with Image Reconstruction and Attention-weighted Pooling