Deep Learning Approaches on Image Captioning: A Review

Abstract

Image captioning is a research area of immense importance, aiming to generate natural language descriptions for visual content in the form of still images. The advent of deep learning and more recently vision-language pre-training techniques has revolutionized the field, leading to more sophisticated methods and improved performance. In this survey paper, we provide a structured review of deep learning methods in image captioning by presenting a comprehensive taxonomy and discussing each method category in detail. Additionally, we examine the datasets commonly employed in image captioning research, as well as the evaluation metrics used to assess the performance of different captioning models. We address the challenges faced in this field by emphasizing issues such as object hallucination, missing context, illumination conditions, contextual understanding, and referring expressions. We rank different deep learning methods' performance according to widely used evaluation metrics, giving insight into the current state of the art. Furthermore, we identify several potential future directions for research in this area, which include tackling the information misalignment problem between image and text modalities, mitigating dataset bias, incorporating vision-language pre-training methods to enhance caption generation, and developing improved evaluation tools to accurately measure the quality of image captions.

41 pages, 6 figures

References in corpus (12)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Conditional Generative Adversarial Nets
• Deformable DETR: Deformable Transformers for End-to-End Object Detection
• Flamingo: a Visual Language Model for Few-Shot Learning
• BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
• Prefix-Tuning: Optimizing Continuous Prompts for Generation
• CPTR: Full Transformer Network for Image Captioning
• A Thorough Review on Recent Deep Learning Methodologies for Image Captioning
• VIVO: Visual Vocabulary Pre-Training for Novel Object Captioning
• Prismer: A Vision-Language Model with Multi-Task Experts
• Image Captioning as an Assistive Technology: Lessons Learned from VizWiz 2020 Challenge
• Image Captioning In the Transformer Age

Cited by in corpus (5)

• An Ensemble Model with Attention Based Mechanism for Image Captioning
• A Comprehensive Survey of 3D Dense Captioning: Localizing and Describing Objects in 3D Scenes
• Attention-based transformer models for image captioning across languages: An in-depth survey and evaluation
• Siamese-Driven Optimization for Low-Resolution Image Latent Embedding in Image Captioning
• Automatic Identification and Description of Jewelry Through Computer Vision and Neural Networks for Translators and Interpreters