3 citations · 8 across the 24 of their papers we have counts for
9 papers · 1 filter
MindEval: Benchmarking Language Models on Multi-turn Mental Health Support
José Pombal, Maya D'Eon, Nuno M. Guerreiro +3
Demand for mental health support through AI chatbots is surging, though current systems present several limitations, like sycophancy or overvalidation, and reinforcement of maladap…
Tower+: Bridging Generality and Translation Specialization in Multilingual LLMs
Ricardo Rei, Nuno M. Guerreiro, José Pombal +4
Fine-tuning pretrained LLMs has been shown to be an effective strategy for reaching state-of-the-art performance on specific tasks like machine translation. However, this process o…
EuroLLM-9B: Technical Report
Pedro Henrique Martins, João Alves, Patrick Fernandes +14
This report presents EuroLLM-9B, a large language model trained from scratch to support the needs of European citizens by covering all 24 official European Union languages and 11 a…
M-Prometheus: A Suite of Open Multilingual LLM Judges
José Pombal, Dongkeun Yoon, Patrick Fernandes +5
The use of language models for automatically evaluating long-form text (LLM-as-a-judge) is becoming increasingly common, yet most LLM judges are optimized exclusively for English,…
Zero-shot Benchmarking: A Framework for Flexible and Scalable Automatic Evaluation of Language Models
José Pombal, Nuno M. Guerreiro, Ricardo Rei +1
As language models improve and become capable of performing more complex tasks across modalities, evaluating them automatically becomes increasingly challenging. Developing strong…
XL-Suite: Cross-Lingual Synthetic Training and Evaluation Data for Open-Ended Generation
Vivek Iyer, Pinzhen Chen, Ricardo Rei +1
Cross-lingual open-ended generation - responding in a language different from that of the query - is an important yet understudied problem. This work proposes XL-Instruct, a novel…
Adding Chocolate to Mint: Mitigating Metric Interference in Machine Translation
José Pombal, Nuno M. Guerreiro, Ricardo Rei +1
As automatic metrics become increasingly stronger and widely adopted, the risk of unintentionally "gaming the metric" during model development rises. This issue is caused by metric…
EuroBERT: Scaling Multilingual Encoders for European Languages
Nicolas Boizard, Hippolyte Gisserot-Boukhlef, Duarte M. Alves +16
General-purpose multilingual vector representations, used in retrieval, regression and classification, are traditionally obtained from bidirectional encoder models. Despite their w…
Translate Smart, not Hard: Cascaded Translation Systems with Quality-Aware Deferral
António Farinhas, Nuno M. Guerreiro, Sweta Agrawal +2
Larger models often outperform smaller ones but come with high computational costs. Cascading offers a potential solution. By default, it uses smaller models and defers only some i…