Longer Version for "Deep Context-Encoding Network for Retinal Image Captioning"
arXiv:2105.14538
Abstract
Automatically generating medical reports for retinal images is one of the promising ways to help ophthalmologists reduce their workload and improve work efficiency. In this work, we propose a new context-driven encoding network to automatically generate medical reports for retinal images. The proposed model is mainly composed of a multi-modal input encoder and a fused-feature decoder. Our experimental results show that our proposed method is capable of effectively leveraging the interactive information between the input image and context, i.e., keywords in our case. The proposed method creates more accurate and meaningful reports for retinal images than baseline models and achieves state-of-the-art performance. This performance is shown in several commonly used metrics for the medical report generation task: BLEU-avg (+16%), CIDEr (+10.2%), and ROUGE (+8.6%).
arXiv commentThis paper is a longer version of "Deep Context-Encoding Network for Retinal Image Captioning" which is accepted by IEEE International Conference on Image Processing (ICIP), 2021
References in corpus (7)
- Improved Image Captioning via Policy Gradient optimization of SPIDEr
- Ask Your Neurons: A Neural-based Approach to Answering Questions about Images
- Generating Visual Explanations
- Characterizing Speech Adversarial Examples Using Self-Attention U-Net Enhancement
- VQABQ: Visual Question Answering by Basic Questions
- PATE-AAE: Incorporating Adversarial Autoencoder into Private Aggregation of Teacher Ensembles for Spoken Command Classification
- Contextualized Keyword Representations for Multi-modal Retinal Image Captioning