Interactive and Explainable Region-guided Radiology Report Generation

Abstract

The automatic generation of radiology reports has the potential to assist radiologists in the time-consuming task of report writing. Existing methods generate the full report from image-level features, failing to explicitly focus on anatomical regions in the image. We propose a simple yet effective region-guided report generation model that detects anatomical regions and then describes individual, salient regions to form the final report. While previous methods generate reports without the possibility of human intervention and with limited explainability, our method opens up novel clinical use cases through additional interactive capabilities and introduces a high degree of transparency and explainability. Comprehensive experiments demonstrate our method's effectiveness in report generation, outperforming previous state-of-the-art models, and highlight its interactive capabilities. The code and checkpoints are available at https://github.com/ttanida/rgrg .

Accepted at CVPR 2023

References in corpus (1)

• Chest ImaGenome Dataset for Clinical Reasoning

Cited by in corpus (12)

• CLIP in Medical Imaging: A Survey
• Automated Radiology Report Generation: A Review of Recent Advances
• The Impact of Imperfect XAI on Human-AI Decision-Making
• Cross-Modal Causal Intervention for Medical Report Generation
• From large language models to multimodal AI: A scoping review on the potential of generative AI in medicine
• PadChest-GR: A Bilingual Chest X-ray Dataset for Grounded Radiology Report Generation
• Automatic Medical Report Generation: Methods and Applications
• M4CXR: Exploring Multi-task Potentials of Multi-modal Large Language Models for Chest X-ray Interpretation
• Structural Entities Extraction and Patient Indications Incorporation for Chest X-ray Report Generation
• GIT-CXR: End-to-End Transformer for Chest X-Ray Report Generation
• Structure Observation Driven Image-Text Contrastive Learning for Computed Tomography Report Generation
• Self-Supervised Anatomical Consistency Learning for Vision-Grounded Medical Report Generation