8 papers
The Grounding Gap: How LLMs Anchor the Meaning of Abstract Concepts Differently from Humans
Odysseas S. Chlapanis, Orfeas Menis Mastromichalakis, Christos H. Papadimitriou
Abstract concepts - justice, theory, availability - have no single perceivable referent; in the human brain, their meaning emerges from a web of experiences, affect, and social con…
Explain the Flag: Contextualizing Hate Speech Beyond Censorship
Jason Liartis, Eirini Kaldeli, Lambrini Gyftokosta +2
Hate, derogatory, and offensive speech remains a persistent challenge in online platforms and public discourse. While automated detection systems are widely used, most focus on cen…
Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
Mike A. Merrill, Alexander G. Shaw, Nicholas Carlini +82
AI agents may soon become capable of autonomously completing valuable, long-horizon tasks in diverse domains. Current benchmarks either do not measure real-world tasks, or are not…
GAMBIT+: A Challenge Set for Evaluating Gender Bias in Machine Translation Quality Estimation Metrics
Giorgos Filandrianos, Orfeas Menis Mastromichalakis, Wafaa Mohammed +2
Gender bias in machine translation (MT) systems has been extensively documented, but bias in automatic quality estimation (QE) metrics remains comparatively underexplored. Existing…
Assumed Identities: Quantifying Gender Bias in Machine Translation of Gender-Ambiguous Occupational Terms
Orfeas Menis Mastromichalakis, Giorgos Filandrianos, Maria Symeonaki +1
Machine Translation (MT) systems frequently encounter gender-ambiguous occupational terms, where they must assign gender without explicit contextual cues. While individual translat…
Don't Erase, Inform! Detecting and Contextualizing Harmful Language in Cultural Heritage Collections
Orfeas Menis Mastromichalakis, Jason Liartis, Kristina Rose +2
Cultural Heritage (CH) data hold invaluable knowledge, reflecting the history, traditions, and identities of societies, and shaping our understanding of the past and present. Howev…