Self-supervised Feature Learning via Exploiting Multi-modal Data for Retinal Disease Diagnosis

Abstract

The automatic diagnosis of various retinal diseases from fundus images is important to support clinical decision-making. However, developing such automatic solutions is challenging due to the requirement of a large amount of human-annotated data. Recently, unsupervised/self-supervised feature learning techniques receive a lot of attention, as they do not need massive annotations. Most of the current self-supervised methods are analyzed with single imaging modality and there is no method currently utilize multi-modal images for better results. Considering that the diagnostics of various vitreoretinal diseases can greatly benefit from another imaging modality, e.g., FFA, this paper presents a novel self-supervised feature learning method by effectively exploiting multi-modal data for retinal disease diagnosis. To achieve this, we first synthesize the corresponding FFA modality and then formulate a patient feature-based softmax embedding objective. Our objective learns both modality-invariant features and patient-similarity features. Through this mechanism, the neural network captures the semantically shared information across different modalities and the apparent visual similarity between patients. We evaluate our method on two public benchmark datasets for retinal disease diagnosis. The experimental results demonstrate that our method clearly outperforms other self-supervised feature learning methods and is comparable to the supervised baseline.

arXiv commentIEEE Transactions on Medical Imaging, code is at https://github.com/xmengli999/self_supervised

References in corpus (8)

• Distilling the Knowledge in a Neural Network
• A Simple Framework for Contrastive Learning of Visual Representations
• Momentum Contrast for Unsupervised Visual Representation Learning
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• Contrastive Multiview Coding
• DeepSeeNet: A deep learning model for automated classification of patient-based age-related macular degeneration severity from color fundus photographs
• Improving Cytoarchitectonic Segmentation of Human Brain Areas with Self-supervised Siamese Networks
• Automatic lesion segmentation and Pathological Myopia classification in fundus images