Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology

Abstract

Stain variation is a phenomenon observed when distinct pathology laboratories stain tissue slides that exhibit similar but not identical color appearance. Due to this color shift between laboratories, convolutional neural networks (CNNs) trained with images from one lab often underperform on unseen images from the other lab. Several techniques have been proposed to reduce the generalization error, mainly grouped into two categories: stain color augmentation and stain color normalization. The former simulates a wide variety of realistic stain variations during training, producing stain-invariant CNNs. The latter aims to match training and test color distributions in order to reduce stain variation. For the first time, we compared some of these techniques and quantified their effect on CNN classification performance using a heterogeneous dataset of hematoxylin and eosin histopathology images from 4 organs and 9 pathology laboratories. Additionally, we propose a novel unsupervised method to perform stain color normalization using a neural network. Based on our experimental results, we provide practical guidelines on how to use stain color augmentation and stain color normalization in future computational pathology applications.

Accepted in the Medical Image Analysis journal

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Auto-Encoding Variational Bayes
• scikit-image: Image processing in Python
• Striving for Simplicity: The All Convolutional Net
• Domain-adversarial neural networks to address the appearance variability of histopathology images
• Machine learning methods for histopathological image analysis
• Detecting Cancer Metastases on Gigapixel Pathology Images
• Predicting breast tumor proliferation from whole-slide images: the TUPAC16 challenge
• Whole-Slide Mitosis Detection in H&E Breast Histology Using PHH3 as a Reference to Train Distilled Stain-Invariant Convolutional Networks
• Epithelium segmentation using deep learning in H&E-stained prostate specimens with immunohistochemistry as reference standard
• Neural Stain-Style Transfer Learning using GAN for Histopathological Images

Cited by in corpus (37)

• Deep neural network models for computational histopathology: A survey
• Artificial Intelligence for Digital and Computational Pathology
• WILDS: A Benchmark of in-the-Wild Distribution Shifts
• Deep Learning for Computational Cytology: A Survey
• StainNet: a fast and robust stain normalization network
• Augmenting the Pathology Lab: An Intelligent Whole Slide Image Classification System for the Real World
• Self-supervised driven consistency training for annotation efficient histopathology image analysis
• Recommendations on test datasets for evaluating AI solutions in pathology
• Learning domain-agnostic visual representation for computational pathology using medically-irrelevant style transfer augmentation
• Similarity and Quality Metrics for MR Image-To-Image Translation
• Pseudo-Bag Mixup Augmentation for Multiple Instance Learning-Based Whole Slide Image Classification
• AdvMIL: Adversarial Multiple Instance Learning for the Survival Analysis on Whole-Slide Images
• RandStainNA: Learning Stain-Agnostic Features from Histology Slides by Bridging Stain Augmentation and Normalization
• A Closer Look at Domain Shift for Deep Learning in Histopathology
• Survey of XAI in digital pathology
• Multi-domain stain normalization for digital pathology: A cycle-consistent adversarial network for whole slide images
• Objective Diagnosis for Histopathological Images Based on Machine Learning Techniques: Classical Approaches and New Trends
• Learning image representations for anomaly detection: application to discovery of histological alterations in drug development
• Evaluating Generic Auto-ML Tools for Computational Pathology
• Selecting Data Augmentation for Simulating Interventions
• Seeing the random forest through the decision trees. Supporting learning health systems from histopathology with machine learning models: Challenges and opportunities
• Improving Generalization Capability of Deep Learning-Based Nuclei Instance Segmentation by Non-deterministic Train Time and Deterministic Test Time Stain Normalization
• Extending Unsupervised Neural Image Compression With Supervised Multitask Learning
• Performance of externally validated machine learning models based on histopathology images for the diagnosis, classification, prognosis, or treatment outcome prediction in female breast cancer: A systematic review
• Evaluating Deep Learning-based Melanoma Classification using Immunohistochemistry and Routine Histology: A Three Center Study
• Benchmarking common uncertainty estimation methods with histopathological images under domain shift and label noise
• Spectral decoupling allows training transferable neural networks in medical imaging
• HistoCartography: A Toolkit for Graph Analytics in Digital Pathology
• Towards Measuring Domain Shift in Histopathological Stain Translation in an Unsupervised Manner
• Source-Free Adaptation to Measurement Shift via Bottom-Up Feature Restoration
• Intensity augmentation for domain transfer of whole breast segmentation in MRI
• Structure-Preserving Multi-Domain Stain Color Augmentation using Style-Transfer with Disentangled Representations
• Automated Scoring of Nuclear Pleomorphism Spectrum with Pathologist-level Performance in Breast Cancer
• Medulloblastoma Tumor Classification using Deep Transfer Learning with Multi-Scale EfficientNets
• Two-step Domain Adaptation for Mitosis Cell Detection in Histopathology Images
• Generalized Categorisation of Digital Pathology Whole Image Slides using Unsupervised Learning
• Data Alchemy: Mitigating Cross-Site Model Variability Through Test Time Data Calibration