4 papers · 1 filter
MT-LENS: An all-in-one Toolkit for Better Machine Translation Evaluation
Javier García Gilabert, Carlos Escolano, Audrey Mash +2
We introduce MT-LENS, a framework designed to evaluate Machine Translation (MT) systems across a variety of tasks, including translation quality, gender bias detection, added toxic…
The power of Prompts: Evaluating and Mitigating Gender Bias in MT with LLMs
Aleix Sant, Carlos Escolano, Audrey Mash +2
This paper studies gender bias in machine translation through the lens of Large Language Models (LLMs). Four widely-used test sets are employed to benchmark various base LLMs, comp…
Investigating the translation capabilities of Large Language Models trained on parallel data only
Javier García Gilabert, Carlos Escolano, Aleix Sant Savall +4
In recent years, Large Language Models (LLMs) have demonstrated exceptional proficiency across a broad spectrum of Natural Language Processing (NLP) tasks, including Machine Transl…
A Hard Nut to Crack: Idiom Detection with Conversational Large Language Models
Francesca De Luca Fornaciari, Begoña Altuna, Itziar Gonzalez-Dios +1
In this work, we explore idiomatic language processing with Large Language Models (LLMs). We introduce the Idiomatic language Test Suite IdioTS, a new dataset of difficult examples…