Early Melanoma Diagnosis with Sequential Dermoscopic Images

Abstract

Dermatologists often diagnose or rule out early melanoma by evaluating the follow-up dermoscopic images of skin lesions. However, existing algorithms for early melanoma diagnosis are developed using single time-point images of lesions. Ignoring the temporal, morphological changes of lesions can lead to misdiagnosis in borderline cases. In this study, we propose a framework for automated early melanoma diagnosis using sequential dermoscopic images. To this end, we construct our method in three steps. First, we align sequential dermoscopic images of skin lesions using estimated Euclidean transformations, extract the lesion growth region by computing image differences among the consecutive images, and then propose a spatio-temporal network to capture the dermoscopic changes from aligned lesion images and the corresponding difference images. Finally, we develop an early diagnosis module to compute probability scores of malignancy for lesion images over time. We collected 179 serial dermoscopic imaging data from 122 patients to verify our method. Extensive experiments show that the proposed model outperforms other commonly used sequence models. We also compared the diagnostic results of our model with those of seven experienced dermatologists and five registrars. Our model achieved higher diagnostic accuracy than clinicians (63.69% vs. 54.33%, respectively) and provided an earlier diagnosis of melanoma (60.7% vs. 32.7% of melanoma correctly diagnosed on the first follow-up images). These results demonstrate that our model can be used to identify melanocytic lesions that are at high-risk of malignant transformation earlier in the disease process and thereby redefine what is possible in the early detection of melanoma.

References in corpus (12)

• Two-Stream Convolutional Networks for Action Recognition in Videos
• The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions
• A Simple Neural Attentive Meta-Learner
• A deep learning system for differential diagnosis of skin diseases
• Deep Learning in Medical Image Registration: A Survey
• Skin Lesion Analysis toward Melanoma Detection: A Challenge at the International Symposium on Biomedical Imaging (ISBI) 2016, hosted by the International Skin Imaging Collaboration (ISIC)
• Optimization for deep learning: theory and algorithms
• Recent Advances in Deep Learning: An Overview
• Fully Convolutional Neural Networks to Detect Clinical Dermoscopic Features
• Skin Cancer Detection and Tracking using Data Synthesis and Deep Learning
• Recent advances in deep learning applied to skin cancer detection
• What evidence does deep learning model use to classify Skin Lesions?