From Modern CNNs to Vision Transformers: Assessing the Performance, Robustness, and Classification Strategies of Deep Learning Models in Histopathology
arXiv:2204.05044 · doi:10.1016/j.media.2023.102809
Abstract
While machine learning is currently transforming the field of histopathology, the domain lacks a comprehensive evaluation of state-of-the-art models based on essential but complementary quality requirements beyond a mere classification accuracy. In order to fill this gap, we developed a new methodology to extensively evaluate a wide range of classification models, including recent vision transformers, and convolutional neural networks such as: ConvNeXt, ResNet (BiT), Inception, ViT and Swin transformer, with and without supervised or self-supervised pretraining. We thoroughly tested the models on five widely used histopathology datasets containing whole slide images of breast, gastric, and colorectal cancer and developed a novel approach using an image-to-image translation model to assess the robustness of a cancer classification model against stain variations. Further, we extended existing interpretability methods to previously unstudied models and systematically reveal insights of the models' classifications strategies that can be transferred to future model architectures.
arXiv comment14 pages, 6 figures, v2: version accepted by Medical Image Analysis, code available under https://github.com/hhi-aml/histobenchmark
References in corpus (15)
- Automated Design of Deep Learning Methods for Biomedical Image Segmentation
- Methods for Interpreting and Understanding Deep Neural Networks
- A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI
- Deep Visual Domain Adaptation: A Survey
- Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
- Domain Adaptation for Medical Image Analysis: A Survey
- Machine learning methods for histopathological image analysis
- A review of domain adaptation without target labels
- Deep Learning for Whole Slide Image Analysis: An Overview
- Self-supervised representation learning from 12-lead ECG data
- Ground Truth Evaluation of Neural Network Explanations with CLEVR-XAI
- Resolving challenges in deep learning-based analyses of histopathological images using explanation methods
- Explaining by Removing: A Unified Framework for Model Explanation
- Self-supervised driven consistency training for annotation efficient histopathology image analysis
- Robust and Efficient Medical Imaging with Self-Supervision