33 citations · 71 across the 29 of their papers we have counts for
8 papers · 1 filter
Teaching Language Models to Faithfully Express their Uncertainty
Bryan Eikema, Evgenia Ilia, José G. C. de Souza +2
Large language models (LLMs) often miscommunicate their uncertainty: repeated queries can produce divergent answers, yet generated responses are typically unhedged or hedged in way…
Unlocking Latent Discourse Translation in LLMs Through Quality-Aware Decoding
Wafaa Mohammed, Vlad Niculae, Chrysoula Zerva
Large language models (LLMs) have emerged as strong contenders in machine translation.Yet, they still struggle to adequately handle discourse phenomena, such as pronoun resolution…
GAMBIT+: A Challenge Set for Evaluating Gender Bias in Machine Translation Quality Estimation Metrics
Giorgos Filandrianos, Orfeas Menis Mastromichalakis, Wafaa Mohammed +2
Gender bias in machine translation (MT) systems has been extensively documented, but bias in automatic quality estimation (QE) metrics remains comparatively underexplored. Existing…
How Well Do Vision-Language Models Understand Movies?
Emmanouil Zaranis, António Farinhas, Saul Santos +28
Vision-language models (VLMs) can increasingly handle extended contexts and answer questions about long videos--but can they understand the narrative structure of movies as humans…
A Conformal Risk Control Framework for Granular Word Assessment and Uncertainty Calibration of CLIPScore Quality Estimates
Gonçalo Gomes, Bruno Martins, Chrysoula Zerva
This study explores current limitations of learned image captioning evaluation metrics, specifically the lack of granular assessments for errors within captions, and the reliance o…
Not Like Us, Hunty: Measuring Perceptions and Behavioral Effects of Minoritized Anthropomorphic Cues in LLMs
Jeffrey Basoah, Daniel Chechelnitsky, Tao Long +5
As large language models (LLMs) increasingly adapt and personalize to diverse sets of users, there is an increased risk of systems appropriating sociolects, i.e., language styles o…
Rejected Dialects: Biases Against African American Language in Reward Models
Joel Mire, Zubin Trivadi Aysola, Daniel Chechelnitsky +3
Preference alignment via reward models helps build safe, helpful, and reliable large language models (LLMs). However, subjectivity in preference judgments and the lack of represent…
Evaluation of Multilingual Image Captioning: How far can we get with CLIP models?
Gonçalo Gomes, Chrysoula Zerva, Bruno Martins
The evaluation of image captions, looking at both linguistic fluency and semantic correspondence to visual contents, has witnessed a significant effort. Still, despite advancements…