4 papers
MT-LENS: An all-in-one Toolkit for Better Machine Translation Evaluation
Javier García Gilabert, Carlos Escolano, Audrey Mash +2
We introduce MT-LENS, a framework designed to evaluate Machine Translation (MT) systems across a variety of tasks, including translation quality, gender bias detection, added toxic…
The power of Prompts: Evaluating and Mitigating Gender Bias in MT with LLMs
Aleix Sant, Carlos Escolano, Audrey Mash +2
This paper studies gender bias in machine translation through the lens of Large Language Models (LLMs). Four widely-used test sets are employed to benchmark various base LLMs, comp…
Investigating the translation capabilities of Large Language Models trained on parallel data only
Javier García Gilabert, Carlos Escolano, Aleix Sant Savall +4
In recent years, Large Language Models (LLMs) have demonstrated exceptional proficiency across a broad spectrum of Natural Language Processing (NLP) tasks, including Machine Transl…
The Catalan Language CLUB
Carlos Rodriguez-Penagos, Carme Armentano-Oller, Marta Villegas +4
The Catalan Language Understanding Benchmark (CLUB) encompasses various datasets representative of different NLU tasks that enable accurate evaluations of language models, followin…