Learning Hierarchical Attention for Weakly-supervised Chest X-Ray Abnormality Localization and Diagnosis

Abstract

We consider the problem of abnormality localization for clinical applications. While deep learning has driven much recent progress in medical imaging, many clinical challenges are not fully addressed, limiting its broader usage. While recent methods report high diagnostic accuracies, physicians have concerns trusting these algorithm results for diagnostic decision-making purposes because of a general lack of algorithm decision reasoning and interpretability. One potential way to address this problem is to further train these models to localize abnormalities in addition to just classifying them. However, doing this accurately will require a large amount of disease localization annotations by clinical experts, a task that is prohibitively expensive to accomplish for most applications. In this work, we take a step towards addressing these issues by means of a new attention-driven weakly supervised algorithm comprising a hierarchical attention mining framework that unifies activation- and gradient-based visual attention in a holistic manner. Our key algorithmic innovations include the design of explicit ordinal attention constraints, enabling principled model training in a weakly-supervised fashion, while also facilitating the generation of visual-attention-driven model explanations by means of localization cues. On two large-scale chest X-ray datasets (NIH ChestX-ray14 and CheXpert), we demonstrate significant localization performance improvements over the current state of the art while also achieving competitive classification performance. Our code is available on https://github.com/oyxhust/HAM.

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases
• Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks
• Squeeze-and-Excitation Networks
• Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer
• Diagnose like a Radiologist: Attention Guided Convolutional Neural Network for Thorax Disease Classification
• Weakly Supervised Deep Learning for Thoracic Disease Classification and Localization on Chest X-rays
• Weakly Supervised Medical Diagnosis and Localization from Multiple Resolutions
• Weakly Supervised Universal Fracture Detection in Pelvic X-rays

Cited by in corpus (11)

• Deep Learning for Chest X-ray Analysis: A Survey
• Follow My Eye: Using Gaze to Supervise Computer-Aided Diagnosis
• CLIP in Medical Imaging: A Survey
• Improving deep neural network generalization and robustness to background bias via layer-wise relevance propagation optimization
• Checklist for responsible deep learning modeling of medical images based on COVID-19 detection studies
• ThoraX-PriorNet: A Novel Attention-Based Architecture Using Anatomical Prior Probability Maps for Thoracic Disease Classification
• Cross- and Intra-image Prototypical Learning for Multi-label Disease Diagnosis and Interpretation
• Artificially Generated Visual Scanpath Improves Multi-label Thoracic Disease Classification in Chest X-Ray Images
• Faster ISNet for Background Bias Mitigation on Deep Neural Networks
• A Hierarchical Slice Attention Network for Appendicitis Classification in 3D CT Scans
• OXnet: Omni-supervised Thoracic Disease Detection from Chest X-rays