Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classification

Abstract

Convolutional Neural Networks (CNN) are state-of-the-art models for many image classification tasks. However, to recognize cancer subtypes automatically, training a CNN on gigapixel resolution Whole Slide Tissue Images (WSI) is currently computationally impossible. The differentiation of cancer subtypes is based on cellular-level visual features observed on image patch scale. Therefore, we argue that in this situation, training a patch-level classifier on image patches will perform better than or similar to an image-level classifier. The challenge becomes how to intelligently combine patch-level classification results and model the fact that not all patches will be discriminative. We propose to train a decision fusion model to aggregate patch-level predictions given by patch-level CNNs, which to the best of our knowledge has not been shown before. Furthermore, we formulate a novel Expectation-Maximization (EM) based method that automatically locates discriminative patches robustly by utilizing the spatial relationships of patches. We apply our method to the classification of glioma and non-small-cell lung carcinoma cases into subtypes. The classification accuracy of our method is similar to the inter-observer agreement between pathologists. Although it is impossible to train CNNs on WSIs, we experimentally demonstrate using a comparable non-cancer dataset of smaller images that a patch-based CNN can outperform an image-based CNN.

References in corpus (6)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
• Fully Convolutional Multi-Class Multiple Instance Learning
• DFDL: Discriminative Feature-oriented Dictionary Learning for Histopathological Image Classification
• From Image-level to Pixel-level Labeling with Convolutional Networks

Cited by in corpus (7)

• Machine learning methods for histopathological image analysis
• Deep Multi-instance Networks with Sparse Label Assignment for Whole Mammogram Classification
• Seeing the random forest through the decision trees. Supporting learning health systems from histopathology with machine learning models: Challenges and opportunities
• Learning and Interpreting Multi-Multi-Instance Learning Networks
• Histopathological Image Classification using Discriminative Feature-oriented Dictionary Learning
• Deep Learning for Automated Medical Image Analysis
• Learning scale-variant and scale-invariant features for deep image classification