13 papers
IndicQE-APE: A Benchmark for Quality Estimation and Automatic Post-Editing for Indic Languages
Diptesh Kanojia, Archchana Sindhujan, Sourabh Deoghare +15
Indic quality estimation (QE) and automatic post-editing (APE) data is spread across separate releases, so no single resource supports training and evaluation across tasks and lang…
SemEval-2026 Task 6: CLARITY -- Unmasking Political Question Evasions
Konstantinos Thomas, Giorgos Filandrianos, Maria Lymperaiou +2
Political speakers often avoid answering questions directly while maintaining the appearance of responsiveness. Despite its importance for public discourse, such strategic evasion…
BEiTScore: Reference-free Image Captioning Evaluation with an Efficient Cross-Encoder Model
Gonçalo Gomes, Bruno Martins, Chrysoula Zerva
Image captioning evaluation remains a significant challenge, as vision-language models evolve toward more challenging capabilities such as generating long-form and context-rich des…
Reasoning or Pattern Matching? Probing Large Vision-Language Models with Visual Puzzles
Maria Lymperaiou, Vasileios Karampinis, Giorgos Filandrianos +3
Puzzles have long served as compact and revealing probes of human cognition, isolating abstraction, rule discovery, and systematic reasoning with minimal reliance on prior knowledg…
Counterfactual Fairness with Graph Uncertainty
Davi Valério, Chrysoula Zerva, Mariana Pinto +2
Evaluating machine learning (ML) model bias is key to building trustworthy and robust ML systems. Counterfactual Fairness (CF) audits allow the measurement of bias of ML models wit…
Teaching Language Models to Faithfully Express their Uncertainty
Bryan Eikema, Evgenia Ilia, José G. C. de Souza +2
Large language models (LLMs) often miscommunicate their uncertainty: repeated queries can produce divergent answers, yet generated responses are typically unhedged or hedged in way…