Building medical image classifiers with very limited data using segmentation networks

Abstract

Deep learning has shown promising results in medical image analysis, however, the lack of very large annotated datasets confines its full potential. Although transfer learning with ImageNet pre-trained classification models can alleviate the problem, constrained image sizes and model complexities can lead to unnecessary increase in computational cost and decrease in performance. As many common morphological features are usually shared by different classification tasks of an organ, it is greatly beneficial if we can extract such features to improve classification with limited samples. Therefore, inspired by the idea of curriculum learning, we propose a strategy for building medical image classifiers using features from segmentation networks. By using a segmentation network pre-trained on similar data as the classification task, the machine can first learn the simpler shape and structural concepts before tackling the actual classification problem which usually involves more complicated concepts. Using our proposed framework on a 3D three-class brain tumor type classification problem, we achieved 82% accuracy on 191 testing samples with 91 training samples. When applying to a 2D nine-class cardiac semantic level classification problem, we achieved 86% accuracy on 263 testing samples with 108 training samples. Comparisons with ImageNet pre-trained classifiers and classifiers trained from scratch are presented.

arXiv commentThis paper was accepted by Medical Image Analysis

References in corpus (11)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Deep Learning in Neural Networks: An Overview
• A Survey on Deep Learning in Medical Image Analysis
• Conditional Generative Adversarial Nets
• How transferable are features in deep neural networks?
• Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
• Training Very Deep Networks
• Deeply-Supervised Nets
• Building Disease Detection Algorithms with Very Small Numbers of Positive Samples

Cited by in corpus (5)

• Cats or CAT scans: transfer learning from natural or medical image source datasets?
• Cross-modality (CT-MRI) prior augmented deep learning for robust lung tumor segmentation from small MR datasets
• Learning to segment images with classification labels
• Dense Pixel-Labeling for Reverse-Transfer and Diagnostic Learning on Lung Ultrasound for COVID-19 and Pneumonia Detection
• Revisiting Hidden Representations in Transfer Learning for Medical Imaging