33 citations · 71 across the 29 of their papers we have counts for
23 papers · 1 filter
CultureBot: A Framework for Retrieval-Augmented Exploration of Cultural Heritage Data
Vasileios Karampinis, Angelos Vlachos, Giorgos Filandrianos +4
Digital cultural heritage collections encode rich relations among objects, places, periods, providers, images, and source institutions, yet users typically access them through keyw…
ARCCS: An Automated Regulatory Compliance Checking System
Giorgos Filandrianos, José Menezes, Chrysoula Zerva +1
Regulatory compliance checking - deciding whether a target document satisfies the obligations of a regulation - requires interpreting dense legal text, identifying which provisions…
Benchmarking Gender Bias in Machine Translation Evaluation Metrics across Occupations
Orfeas Menis Mastromichalakis, Giorgos Filandrianos, Wafaa Mohammed +2
Gender bias remains a persistent concern in machine translation (MT), affecting both generated translations and their automatic evaluation. When a source text leaves a person's gen…
IndicQE-APE: A Consolidated Benchmark for Quality Estimation and Automatic Post-Editing for Indic Languages
Diptesh Kanojia, Archchana Sindhujan, Sourabh Deoghare +14
Indic quality estimation (QE) and automatic post-editing (APE) data is spread across separate releases, so no single resource supports training and evaluation across tasks and lang…
SemEval-2026 Task 6: CLARITY -- Unmasking Political Question Evasions
Konstantinos Thomas, Giorgos Filandrianos, Maria Lymperaiou +2
Political speakers often avoid answering questions directly while maintaining the appearance of responsiveness. Despite its importance for public discourse, such strategic evasion…
Teaching Language Models to Faithfully Express their Uncertainty
Bryan Eikema, Evgenia Ilia, José G. C. de Souza +2
Large language models (LLMs) often miscommunicate their uncertainty: repeated queries can produce divergent answers, yet generated responses are typically unhedged or hedged in way…
Unlocking Latent Discourse Translation in LLMs Through Quality-Aware Decoding
Wafaa Mohammed, Vlad Niculae, Chrysoula Zerva
Large language models (LLMs) have emerged as strong contenders in machine translation.Yet, they still struggle to adequately handle discourse phenomena, such as pronoun resolution…
GAMBIT+: A Challenge Set for Evaluating Gender Bias in Machine Translation Quality Estimation Metrics
Giorgos Filandrianos, Orfeas Menis Mastromichalakis, Wafaa Mohammed +2
Gender bias in machine translation (MT) systems has been extensively documented, but bias in automatic quality estimation (QE) metrics remains comparatively underexplored. Existing…
A Conformal Risk Control Framework for Granular Word Assessment and Uncertainty Calibration of CLIPScore Quality Estimates
Gonçalo Gomes, Bruno Martins, Chrysoula Zerva
This study explores current limitations of learned image captioning evaluation metrics, specifically the lack of granular assessments for errors within captions, and the reliance o…
Rejected Dialects: Biases Against African American Language in Reward Models
Joel Mire, Zubin Trivadi Aysola, Daniel Chechelnitsky +3
Preference alignment via reward models helps build safe, helpful, and reliable large language models (LLMs). However, subjectivity in preference judgments and the lack of represent…
Evaluation of Multilingual Image Captioning: How far can we get with CLIP models?
Gonçalo Gomes, Chrysoula Zerva, Bruno Martins
The evaluation of image captions, looking at both linguistic fluency and semantic correspondence to visual contents, has witnessed a significant effort. Still, despite advancements…
"I Never Said That": A dataset, taxonomy and baselines on response clarity classification
Konstantinos Thomas, Giorgos Filandrianos, Maria Lymperaiou +2
Equivocation and ambiguity in public speech are well-studied discourse phenomena, especially in political science and analysis of political interviews. Inspired by the well-grounde…
Conformal Prediction for Natural Language Processing: A Survey
Margarida M. Campos, António Farinhas, Chrysoula Zerva +2
The rapid proliferation of large language models and natural language processing (NLP) applications creates a crucial need for uncertainty quantification to mitigate risks such as…
Non-Exchangeable Conformal Language Generation with Nearest Neighbors
Dennis Ulmer, Chrysoula Zerva, André F. T. Martins
Quantifying uncertainty in automatically generated text is important for letting humans check potential hallucinations and making systems more reliable. Conformal prediction is an…
Context-aware Neural Machine Translation for English-Japanese Business Scene Dialogues
Sumire Honda, Patrick Fernandes, Chrysoula Zerva
Despite the remarkable advancements in machine translation, the current sentence-level paradigm faces challenges when dealing with highly-contextual languages like Japanese. In thi…
Uncertainty in Natural Language Generation: From Theory to Applications
Joris Baan, Nico Daheim, Evgenia Ilia +7
Recent advances of powerful Language Models have allowed Natural Language Generation (NLG) to emerge as an important technology that can not only perform traditional tasks like sum…
Conformalizing Machine Translation Evaluation
Chrysoula Zerva, André F. T. Martins
Several uncertainty estimation methods have been recently proposed for machine translation evaluation. While these methods can provide a useful indication of when not to trust mode…
BLEU Meets COMET: Combining Lexical and Neural Metrics Towards Robust Machine Translation Evaluation
Taisiya Glushkova, Chrysoula Zerva, André F. T. Martins
Although neural-based machine translation evaluation metrics, such as COMET or BLEURT, have achieved strong correlations with human judgements, they are sometimes unreliable in det…
Counterfactuals of Counterfactuals: a back-translation-inspired approach to analyse counterfactual editors
Giorgos Filandrianos, Edmund Dervakos, Orfeas Menis-Mastromichalakis +2
In the wake of responsible AI, interpretability methods, which attempt to provide an explanation for the predictions of neural models have seen rapid progress. In this work, we are…
CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Task
Ricardo Rei, Marcos Treviso, Nuno M. Guerreiro +9
We present the joint contribution of IST and Unbabel to the WMT 2022 Shared Task on Quality Estimation (QE). Our team participated on all three subtasks: (i) Sentence and Word-leve…
Disentangling Uncertainty in Machine Translation Evaluation
Chrysoula Zerva, Taisiya Glushkova, Ricardo Rei +1
Trainable evaluation metrics for machine translation (MT) exhibit strong correlation with human judgements, but they are often hard to interpret and might produce unreliable scores…
Learning Disentangled Representations of Negation and Uncertainty
Jake Vasilakes, Chrysoula Zerva, Makoto Miwa +1
Negation and uncertainty modeling are long-standing tasks in natural language processing. Linguistic theory postulates that expressions of negation and uncertainty are semantically…
Uncertainty-Aware Machine Translation Evaluation
Taisiya Glushkova, Chrysoula Zerva, Ricardo Rei +1
Several neural-based metrics have been recently proposed to evaluate machine translation quality. However, all of them resort to point estimates, which provide limited information…