14 papers
Translating in style: An open-weight alternative to GPT-4 for culturally aligned machine translation
Thomas Rietman, Hadi Mohammadi, Robert A. Bagheri
Machine translation systems often preserve meaning but systematically flatten linguistic style, a failure that is most acute when translating into non-Western languages. The RASTA…
trajectory-judge: What Outcome-Only LLM Judges Miss on Agent Trajectories
Hadi Mohammadi
A direct test of an LLM judge of agent trajectories injects faults into correct runs and reports recall, per fault type or by whether the fault broke the environment outcome (loud)…
Learning Sexism Detection Using Multi-Agent Perspectivist Preference Optimization
Hadi Mohammadi, Tina Shahedi, Robert A. Bagheri +2
When people label text for sexism, they often disagree, and not because some of them are wrong: they genuinely perceive sexism differently. Most NLP systems discard this disagreeme…
Reliability-Aware Sexism Detection: Combining DPO with Annotator Agreement and Token-Level Confidence Scoring
Hadi Mohammadi, Shihan Wang, Masoume M. Raeissi +1
The detection of online sexism remains an open problem. Sexism detection is inherently subjective, yet most existing systems reduce multi-annotator labels to a single majority deci…
Evaluating GRPO and DPO for Faithful Chain-of-Thought Reasoning in LLMs
Hadi Mohammadi, Tamas Kozak, Anastasia Giachanou
Chain-of-thought (CoT) reasoning has emerged as a powerful technique for improving the problem-solving capabilities of large language models (LLMs), particularly for tasks requirin…
EvalMORAAL: Interpretable Chain-of-Thought and LLM-as-Judge Evaluation for Moral Alignment in Large Language Models
Hadi Mohammadi, Anastasia Giachanou, Robert A. Bagheri
We present EvalMORAAL, a transparent chain-of-thought (CoT) framework that uses two scoring methods (log-probabilities and direct ratings) plus a model-as-judge peer review to eval…
Do Large Language Models Understand Morality Across Cultures?
Hadi Mohammadi, Yasmeen F. S. S. Meijer, Efthymia Papadopoulou +1
Recent advancements in large language models (LLMs) have established them as powerful tools across numerous domains. However, persistent concerns about embedded biases, such as gen…
Assessing the Reliability of LLMs Annotations in the Context of Demographic Bias and Model Explanation
Hadi Mohammadi, Tina Shahedi, Pablo Mosteiro +3
Understanding the sources of variability in annotations is crucial for developing fair NLP systems, especially for tasks like sexism detection where demographic bias is a concern.…
Exploring Cultural Variations in Moral Judgments with Large Language Models
Hadi Mohammadi, Ayoub Bagheri
Large Language Models (LLMs) have shown strong performance across many tasks, but their ability to capture culturally diverse moral values remains unclear. In this paper, we examin…
Explainability-Based Token Replacement on LLM-Generated Text
Hadi Mohammadi, Anastasia Giachanou, Daniel L. Oberski +1
Generative models, especially large language models (LLMs), have shown remarkable progress in producing text that appears human-like. However, they often exhibit patterns that make…
Anchor-based oversampling for imbalanced tabular data via contrastive and adversarial learning
Hadi Mohammadi, Ehsan Nazerfard, Mostafa Haghir Chehreghani
Imbalanced data represent a distribution with more frequencies of one class (majority) than the other (minority). This phenomenon occurs across various domains, such as security, m…
Explainability in Practice: A Survey of Explainable NLP Across Various Domains
Hadi Mohammadi, Robert A. Bagheri, Anastasia Giachanou +1
Natural Language Processing (NLP) is now embedded in critical sectors including healthcare, finance, and customer relationship management, where models such as GPT-4o, Gemini, and…
LLMs as mirrors of societal moral standards: reflection of cultural divergence and agreement across ethical topics
Mijntje Meijer, Hadi Mohammadi, Ayoub Bagheri
Large language models (LLMs) have become increasingly pivotal in various domains due the recent advancements in their performance capabilities. However, concerns persist regarding…
Large Language Models as Mirrors of Societal Moral Standards
Evi Papadopoulou, Hadi Mohammadi, Ayoub Bagheri
Prior research has demonstrated that language models can, to a limited extent, represent moral norms in a variety of cultural contexts. This research aims to replicate these findin…