3 citations · 3 across the 20 of their papers we have counts for
21 papers
Benchmarking Gender Bias in Machine Translation Evaluation Metrics across Occupations
Orfeas Menis Mastromichalakis, Giorgos Filandrianos, Wafaa Mohammed +2
Gender bias remains a persistent concern in machine translation (MT), affecting both generated translations and their automatic evaluation. When a source text leaves a person's gen…
An Empirical Study of Counterfactual Self-Explanations in LLMs
Giannis Kalyvas, Giorgos Filandrianos, Orfeas Menis Mastromichalakis +2
Large language models can easily generate explanations for their own outputs, but such self-explanations are not necessarily faithful to the model's behavior. We study this issue t…
Harbor Adapters and Harbor-Index: Infrastructure and a Curated Meta-Dataset for Large-Scale Agentic Evaluation
Lin Shi, Haowei Lin, Zixuan Zhu +123
Evaluating agents on the growing number of agentic benchmarks is challenging because they often require complex environments and agent integrations. We introduce Harbor Adapters, a…
AURA-Eval: Evaluation Framework for Acting Under Risk Awareness in LLM Agent Trajectories
Ruoxi Shang, Christina-Maria Androna, Orfeas Menis Mastromichalakis +6
LLM agents are increasingly taking actions with real consequences, yet safety evaluations often reduce their behavior to a final safe/unsafe outcome, obscuring whether the agent re…
Last Translation Benchmark
Vilém Zouhar, Niyati Bafna, Mukund Choudhary +263
For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. As models get stronger, stan…
The Grounding Gap: How LLMs Anchor the Meaning of Abstract Concepts Differently from Humans
Odysseas S. Chlapanis, Orfeas Menis Mastromichalakis, Christos H. Papadimitriou
Abstract concepts - justice, theory, availability - have no single perceivable referent; in the human brain, their meaning emerges from a web of experiences, affect, and social con…
Explain the Flag: Contextualizing Hate Speech Beyond Censorship
Jason Liartis, Eirini Kaldeli, Lambrini Gyftokosta +2
Hate, derogatory, and offensive speech remains a persistent challenge in online platforms and public discourse. While automated detection systems are widely used, most focus on cen…
Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
Mike A. Merrill, Alexander G. Shaw, Nicholas Carlini +82
AI agents may soon become capable of autonomously completing valuable, long-horizon tasks in diverse domains. Current benchmarks either do not measure real-world tasks, or are not…
GAMBIT+: A Challenge Set for Evaluating Gender Bias in Machine Translation Quality Estimation Metrics
Giorgos Filandrianos, Orfeas Menis Mastromichalakis, Wafaa Mohammed +2
Gender bias in machine translation (MT) systems has been extensively documented, but bias in automatic quality estimation (QE) metrics remains comparatively underexplored. Existing…
Don't Erase, Inform! Detecting and Contextualizing Harmful Language in Cultural Heritage Collections
Orfeas Menis Mastromichalakis, Jason Liartis, Kristina Rose +2
Cultural Heritage (CH) data hold invaluable knowledge, reflecting the history, traditions, and identities of societies, and shaping our understanding of the past and present. Howev…
AILS-NTUA at SemEval-2025 Task 4: Parameter-Efficient Unlearning for Large Language Models using Data Chunking
Iraklis Premptis, Maria Lymperaiou, Giorgos Filandrianos +3
The Unlearning Sensitive Content from Large Language Models task aims to remove targeted datapoints from trained models while minimally affecting their general knowledge. In our wo…
Assumed Identities: Quantifying Gender Bias in Machine Translation of Gender-Ambiguous Occupational Terms
Orfeas Menis Mastromichalakis, Giorgos Filandrianos, Maria Symeonaki +1
Machine Translation (MT) systems frequently encounter gender-ambiguous occupational terms, where they must assign gender without explicit contextual cues. While individual translat…
GOSt-MT: A Knowledge Graph for Occupation-related Gender Biases in Machine Translation
Orfeas Menis Mastromichalakis, Giorgos Filandrianos, Eva Tsouparopoulou +3
Gender bias in machine translation (MT) systems poses significant challenges that often result in the reinforcement of harmful stereotypes. Especially in the labour domain where fr…
MusicLIME: Explainable Multimodal Music Understanding
Theodoros Sotirou, Vassilis Lyberatos, Orfeas Menis Mastromichalakis +1
Multimodal models are critical for music understanding tasks, as they capture the complex interplay between audio and lyrics. However, as these models become more prevalent, the ne…
Semantic Prototypes: Enhancing Transparency Without Black Boxes
Orfeas Menis-Mastromichalakis, Giorgos Filandrianos, Jason Liartis +2
As machine learning (ML) models and datasets increase in complexity, the demand for methods that enhance explainability and interpretability becomes paramount. Prototypes, by encap…
Deep Ensemble Art Style Recognition
Orfeas Menis-Mastromichalakis, Natasa Sofou, Giorgos Stamou
The massive digitization of artworks during the last decades created the need for categorization, analysis, and management of huge amounts of data related to abstract concepts, hig…
Beyond One-Size-Fits-All: Adapting Counterfactual Explanations to User Objectives
Orfeas Menis Mastromichalakis, Jason Liartis, Giorgos Stamou
Explainable Artificial Intelligence (XAI) has emerged as a critical area of research aimed at enhancing the transparency and interpretability of AI systems. Counterfactual Explanat…
Employing Crowdsourcing for Enriching a Music Knowledge Base in Higher Education
Vassilis Lyberatos, Spyridon Kantarelis, Eirini Kaldeli +4
This paper describes the methodology followed and the lessons learned from employing crowdsourcing techniques as part of a homework assignment involving higher education students o…
Counterfactuals of Counterfactuals: a back-translation-inspired approach to analyse counterfactual editors
Giorgos Filandrianos, Edmund Dervakos, Orfeas Menis-Mastromichalakis +2
In the wake of responsible AI, interpretability methods, which attempt to provide an explanation for the predictions of neural models have seen rapid progress. In this work, we are…
Towards explainable evaluation of language models on the semantic similarity of visual concepts
Maria Lymperaiou, George Manoliadis, Orfeas Menis Mastromichalakis +2
Recent breakthroughs in NLP research, such as the advent of Transformer models have indisputably contributed to major advancements in several tasks. However, few works research rob…
Computing Rule-Based Explanations of Machine Learning Classifiers using Knowledge Graphs
Edmund Dervakos, Orfeas Menis-Mastromichalakis, Alexandros Chortaras +1
The use of symbolic knowledge representation and reasoning as a way to resolve the lack of transparency of machine learning classifiers is a research area that lately attracts many…