4 papers
BEiTScore: Reference-free Image Captioning Evaluation with an Efficient Cross-Encoder Model
Gonçalo Gomes, Bruno Martins, Chrysoula Zerva
Image captioning evaluation remains a significant challenge, as vision-language models evolve toward more challenging capabilities such as generating long-form and context-rich des…
Text-Only Training for Image Captioning with Retrieval Augmentation and Modality Gap Correction
Rui Fonseca, Bruno Martins, Gil Rocha
Image captioning has drawn considerable attention from the natural language processing and computer vision fields. Aiming to reduce the reliance on curated data, several studies ha…
A Conformal Risk Control Framework for Granular Word Assessment and Uncertainty Calibration of CLIPScore Quality Estimates
Gonçalo Gomes, Bruno Martins, Chrysoula Zerva
This study explores current limitations of learned image captioning evaluation metrics, specifically the lack of granular assessments for errors within captions, and the reliance o…
Evaluation of Multilingual Image Captioning: How far can we get with CLIP models?
Gonçalo Gomes, Chrysoula Zerva, Bruno Martins
The evaluation of image captions, looking at both linguistic fluency and semantic correspondence to visual contents, has witnessed a significant effort. Still, despite advancements…