Tissue Concepts: supervised foundation models in computational pathology

Abstract

Due to the increasing workload of pathologists, the need for automation to support diagnostic tasks and quantitative biomarker evaluation is becoming more and more apparent. Foundation models have the potential to improve generalizability within and across centers and serve as starting points for data efficient development of specialized yet robust AI models. However, the training foundation models themselves is usually very expensive in terms of data, computation, and time. This paper proposes a supervised training method that drastically reduces these expenses. The proposed method is based on multi-task learning to train a joint encoder, by combining 16 different classification, segmentation, and detection tasks on a total of 912,000 patches. Since the encoder is capable of capturing the properties of the samples, we term it the Tissue Concepts encoder. To evaluate the performance and generalizability of the Tissue Concepts encoder across centers, classification of whole slide images from four of the most prevalent solid cancers - breast, colon, lung, and prostate - was used. The experiments show that the Tissue Concepts model achieve comparable performance to models trained with self-supervision, while requiring only 6% of the amount of training patches. Furthermore, the Tissue Concepts encoder outperforms an ImageNet pre-trained encoder on both in-domain and out-of-domain data.

22 Pages, 3 Figures, submitted to and under revision at Computers in Biology and Medicine

References in corpus (17)

• Decoupled Weight Decay Regularization
• On the Opportunities and Risks of Foundation Models
• DINOv2: Learning Robust Visual Features without Supervision
• A deep learning system for differential diagnosis of skin diseases
• BACH: Grand Challenge on Breast Cancer Histology Images
• Segment Anything Model for Medical Images?
• Metrics reloaded: Recommendations for image analysis validation
• MILD-Net: Minimal Information Loss Dilated Network for Gland Instance Segmentation in Colon Histology Images
• Towards artificial general intelligence via a multimodal foundation model
• Neural Image Compression for Gigapixel Histopathology Image Analysis
• Mitosis domain generalization in histopathology images -- The MIDOG challenge
• Epithelium segmentation using deep learning in H&E-stained prostate specimens with immunohistochemistry as reference standard
• Multi-task pre-training of deep neural networks for digital pathology
• AdvMIL: Adversarial Multiple Instance Learning for the Survival Analysis on Whole-Slide Images
• Computational Pathology at Health System Scale -- Self-Supervised Foundation Models from Three Billion Images
• TIAger: Tumor-Infiltrating Lymphocyte Scoring in Breast Cancer for the TiGER Challenge
• Weakly-Supervised Deep Learning Model for Prostate Cancer Diagnosis and Gleason Grading of Histopathology Images