33 citations · 71 across the 29 of their papers we have counts for
31 papers
CultureBot: A Framework for Retrieval-Augmented Exploration of Cultural Heritage Data
Vasileios Karampinis, Angelos Vlachos, Giorgos Filandrianos +4
Digital cultural heritage collections encode rich relations among objects, places, periods, providers, images, and source institutions, yet users typically access them through keyw…
ARCCS: An Automated Regulatory Compliance Checking System
Giorgos Filandrianos, José Menezes, Chrysoula Zerva +1
Regulatory compliance checking - deciding whether a target document satisfies the obligations of a regulation - requires interpreting dense legal text, identifying which provisions…
Benchmarking Gender Bias in Machine Translation Evaluation Metrics across Occupations
Orfeas Menis Mastromichalakis, Giorgos Filandrianos, Wafaa Mohammed +2
Gender bias remains a persistent concern in machine translation (MT), affecting both generated translations and their automatic evaluation. When a source text leaves a person's gen…
AURA-Eval: Evaluation Framework for Acting Under Risk Awareness in LLM Agent Trajectories
Ruoxi Shang, Christina-Maria Androna, Orfeas Menis Mastromichalakis +6
LLM agents are increasingly taking actions with real consequences, yet safety evaluations often reduce their behavior to a final safe/unsafe outcome, obscuring whether the agent re…
IndicQE-APE: A Consolidated Benchmark for Quality Estimation and Automatic Post-Editing for Indic Languages
Diptesh Kanojia, Archchana Sindhujan, Sourabh Deoghare +14
Indic quality estimation (QE) and automatic post-editing (APE) data is spread across separate releases, so no single resource supports training and evaluation across tasks and lang…
BEiTScore: Reference-free Image Captioning Evaluation with an Efficient Cross-Encoder Model
Gonçalo Gomes, Bruno Martins, Chrysoula Zerva
Image captioning evaluation remains a significant challenge, as vision-language models evolve toward more challenging capabilities such as generating long-form and context-rich des…
Locating Translation as a Craft in the Age of AI
Daniel Chechelnitsky, Sireesh Gururaja, Seyi Olojo +4
Rapid development of Large Language Models (LLMs) and similar automated approaches for translation tasks is increasingly affecting the landscape of translation technologies. As con…
SemEval-2026 Task 6: CLARITY -- Unmasking Political Question Evasions
Konstantinos Thomas, Giorgos Filandrianos, Maria Lymperaiou +2
Political speakers often avoid answering questions directly while maintaining the appearance of responsiveness. Despite its importance for public discourse, such strategic evasion…
Reasoning or Pattern Matching? Probing Large Vision-Language Models with Visual Puzzles
Maria Lymperaiou, Vasileios Karampinis, Giorgos Filandrianos +3
Puzzles have long served as compact and revealing probes of human cognition, isolating abstraction, rule discovery, and systematic reasoning with minimal reliance on prior knowledg…
Counterfactual Fairness with Graph Uncertainty
Davi Valério, Chrysoula Zerva, Mariana Pinto +2
Evaluating machine learning (ML) model bias is key to building trustworthy and robust ML systems. Counterfactual Fairness (CF) audits allow the measurement of bias of ML models wit…
Teaching Language Models to Faithfully Express their Uncertainty
Bryan Eikema, Evgenia Ilia, José G. C. de Souza +2
Large language models (LLMs) often miscommunicate their uncertainty: repeated queries can produce divergent answers, yet generated responses are typically unhedged or hedged in way…
Unlocking Latent Discourse Translation in LLMs Through Quality-Aware Decoding
Wafaa Mohammed, Vlad Niculae, Chrysoula Zerva
Large language models (LLMs) have emerged as strong contenders in machine translation.Yet, they still struggle to adequately handle discourse phenomena, such as pronoun resolution…
GAMBIT+: A Challenge Set for Evaluating Gender Bias in Machine Translation Quality Estimation Metrics
Giorgos Filandrianos, Orfeas Menis Mastromichalakis, Wafaa Mohammed +2
Gender bias in machine translation (MT) systems has been extensively documented, but bias in automatic quality estimation (QE) metrics remains comparatively underexplored. Existing…
How Well Do Vision-Language Models Understand Movies?
Emmanouil Zaranis, António Farinhas, Saul Santos +28
Vision-language models (VLMs) can increasingly handle extended contexts and answer questions about long videos--but can they understand the narrative structure of movies as humans…
A Conformal Risk Control Framework for Granular Word Assessment and Uncertainty Calibration of CLIPScore Quality Estimates
Gonçalo Gomes, Bruno Martins, Chrysoula Zerva
This study explores current limitations of learned image captioning evaluation metrics, specifically the lack of granular assessments for errors within captions, and the reliance o…
Not Like Us, Hunty: Measuring Perceptions and Behavioral Effects of Minoritized Anthropomorphic Cues in LLMs
Jeffrey Basoah, Daniel Chechelnitsky, Tao Long +5
As large language models (LLMs) increasingly adapt and personalize to diverse sets of users, there is an increased risk of systems appropriating sociolects, i.e., language styles o…
Rejected Dialects: Biases Against African American Language in Reward Models
Joel Mire, Zubin Trivadi Aysola, Daniel Chechelnitsky +3
Preference alignment via reward models helps build safe, helpful, and reliable large language models (LLMs). However, subjectivity in preference judgments and the lack of represent…
Evaluation of Multilingual Image Captioning: How far can we get with CLIP models?
Gonçalo Gomes, Chrysoula Zerva, Bruno Martins
The evaluation of image captions, looking at both linguistic fluency and semantic correspondence to visual contents, has witnessed a significant effort. Still, despite advancements…
"I Never Said That": A dataset, taxonomy and baselines on response clarity classification
Konstantinos Thomas, Giorgos Filandrianos, Maria Lymperaiou +2
Equivocation and ambiguity in public speech are well-studied discourse phenomena, especially in political science and analysis of political interviews. Inspired by the well-grounde…
Conformal Prediction for Natural Language Processing: A Survey
Margarida M. Campos, António Farinhas, Chrysoula Zerva +2
The rapid proliferation of large language models and natural language processing (NLP) applications creates a crucial need for uncertainty quantification to mitigate risks such as…
Non-Exchangeable Conformal Language Generation with Nearest Neighbors
Dennis Ulmer, Chrysoula Zerva, André F. T. Martins
Quantifying uncertainty in automatically generated text is important for letting humans check potential hallucinations and making systems more reliable. Conformal prediction is an…
Context-aware Neural Machine Translation for English-Japanese Business Scene Dialogues
Sumire Honda, Patrick Fernandes, Chrysoula Zerva
Despite the remarkable advancements in machine translation, the current sentence-level paradigm faces challenges when dealing with highly-contextual languages like Japanese. In thi…
Non-Exchangeable Conformal Risk Control
António Farinhas, Chrysoula Zerva, Dennis Ulmer +1
Split conformal prediction has recently sparked great interest due to its ability to provide formally guaranteed uncertainty sets or intervals for predictions made by black-box neu…
Uncertainty in Natural Language Generation: From Theory to Applications
Joris Baan, Nico Daheim, Evgenia Ilia +7
Recent advances of powerful Language Models have allowed Natural Language Generation (NLG) to emerge as an important technology that can not only perform traditional tasks like sum…