BS-Net: learning COVID-19 pneumonia severity on a large Chest X-Ray dataset

Abstract

In this work we design an end-to-end deep learning architecture for predicting, on Chest X-rays images (CXR), a multi-regional score conveying the degree of lung compromise in COVID-19 patients. Such semi-quantitative scoring system, namely Brixia~score, is applied in serial monitoring of such patients, showing significant prognostic value, in one of the hospitals that experienced one of the highest pandemic peaks in Italy. To solve such a challenging visual task, we adopt a weakly supervised learning strategy structured to handle different tasks (segmentation, spatial alignment, and score estimation) trained with a "from-the-part-to-the-whole" procedure involving different datasets. In particular, we exploit a clinical dataset of almost 5,000 CXR annotated images collected in the same hospital. Our BS-Net demonstrates self-attentive behavior and a high degree of accuracy in all processing stages. Through inter-rater agreement tests and a gold standard comparison, we show that our solution outperforms single human annotators in rating accuracy and consistency, thus supporting the possibility of using this tool in contexts of computer-assisted monitoring. Highly resolved (super-pixel level) explainability maps are also generated, with an original technique, to visually help the understanding of the network activity on the lung areas. We also consider other scores proposed in literature and provide a comparison with a recently proposed non-specific approach. We eventually test the performance robustness of our model on an assorted public COVID-19 dataset, for which we also provide Brixia~score annotations, observing good direct generalization and fine-tuning capabilities that highlight the portability of BS-Net in other clinical settings. The CXR dataset along with the source code and the trained model are publicly released for research purposes.

28 pages, 11 figures, preprint of accepted paper to Medical Image Analysis, Project page with Code and Dataset Available at https://brixia.github.io/

References in corpus (18)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Fully Convolutional Networks for Semantic Segmentation
• Albumentations: fast and flexible image augmentations
• Domain-adversarial neural networks to address the appearance variability of histopathology images
• Review of Artificial Intelligence Techniques in Imaging Data Acquisition, Segmentation and Diagnosis for COVID-19
• COVID-19 Image Data Collection: Prospective Predictions Are the Future
• COVID-19 Image Data Collection
• Searching for Activation Functions
• Rapid AI Development Cycle for the Coronavirus (COVID-19) Pandemic: Initial Results for Automated Detection & Patient Monitoring using Deep Learning CT Image Analysis
• COVID-19 identification in chest X-ray images on flat and hierarchical classification scenarios
• Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation
• COVID-Net: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest X-Ray Images
• Unveiling COVID-19 from Chest X-ray with deep learning: a hurdles race with small data
• COVID-19 in CXR: from Detection and Severity Scoring to Patient Disease Monitoring
• COVID-MobileXpert: On-Device COVID-19 Patient Triage and Follow-up using Chest X-rays
• COVID-19: A Survey on Public Medical Imaging Data Resources
• COVID-Net S: Towards computer-aided severity assessment via training and validation of deep neural networks for geographic extent and opacity extent scoring of chest X-rays for SARS-CoV-2 lung disease severity

Cited by in corpus (11)

• Recent Progress in Transformer-based Medical Image Analysis
• Improving deep neural network generalization and robustness to background bias via layer-wise relevance propagation optimization
• Checklist for responsible deep learning modeling of medical images based on COVID-19 detection studies
• Multimodal Explainability via Latent Shift applied to COVID-19 stratification
• How to Assess Trustworthy AI in Practice
• Multi-objective optimization determines when, which and how to fuse deep networks: an application to predict COVID-19 outcomes
• Federated Split Vision Transformer for COVID-19 CXR Diagnosis using Task-Agnostic Training
• Rad4XCNN: a new agnostic method for post-hoc global explanation of CNN-derived features by means of radiomics
• Multi-Dataset Multi-Task Learning for COVID-19 Prognosis
• Benchmarking Foundation Models and Parameter-Efficient Fine-Tuning for Prognosis Prediction in Medical Imaging
• Faster ISNet for Background Bias Mitigation on Deep Neural Networks