Multimodal Generalized Zero Shot Learning for Gleason Grading using Self-Supervised Learning

Abstract

Gleason grading from histopathology images is essential for accurate prostate cancer (PCa) diagnosis. Since such images are obtained after invasive tissue resection quick diagnosis is challenging under the existing paradigm. We propose a method to predict Gleason grades from magnetic resonance (MR) images which are non-interventional and easily acquired. We solve the problem in a generalized zero-shot learning (GZSL) setting since we may not access training images of every disease grade. Synthetic MRI feature vectors of unseen grades (classes) are generated by exploiting Gleason grades' ordered nature through a conditional variational autoencoder (CVAE) incorporating self-supervised learning. Corresponding histopathology features are generated using cycle GANs, and combined with MR features to predict Gleason grades of test images. Experimental results show our method outperforms competing feature generating approaches for GZSL, and comes close to performance of fully supervised methods.

References in corpus (17)

• Automated Gleason Grading of Prostate Biopsies using Deep Learning
• Synergic Adversarial Label Learning for Grading Retinal Diseases via Knowledge Distillation and Multi-task Learning
• Terabyte-scale Deep Multiple Instance Learning for Classification and Localization in Pathology
• Retinal Vasculature Segmentation Using Local Saliency Maps and Generative Adversarial Networks For Image Super Resolution
• Semi-Supervised Histology Classification using Deep Multiple Instance Learning and Contrastive Predictive Coding
• Night Time Haze and Glow Removal using Deep Dilated Convolutional Network
• Chest X-rays Classification: A Multi-Label and Fine-Grained Problem
• Progressive Generative Adversarial Networks for Medical Image Super resolution
• GAN Based Medical Image Registration
• Pathological Retinal Region Segmentation From OCT Images Using Geometric Relation Based Augmentation
• Improving Medical Image Classification with Label Noise Using Dual-uncertainty Estimation
• Registration of Histopathogy Images Using Structural Information From Fine Grained Feature Maps
• sZoom: A Framework for Automatic Zoom into High Resolution Surveillance Videos
• Anomaly Detection on X-Rays Using Self-Supervised Aggregation Learning
• Multi-scale Deep Learning Architecture for Nucleus Detection in Renal Cell Carcinoma Microscopy Image
• Generative Adversarial Networks And Domain Adaptation For Training Data Independent Image Registration
• CT Image Synthesis Using Weakly Supervised Segmentation and Geometric Inter-Label Relations For COVID Image Analysis