33 citations · 83 across the 15 of their papers we have counts for
15 papers
Fine PT-PT Web: A High-Quality 41 Billion Tokens Data Collection of the European Portuguese Web
Gonçalo Vinagre, Rui Pedro Guerra, Pedro Gomes +7
Curating Web corpora for regional language variants like European Portuguese (PT-PT) is heavily bottlenecked by dialectal overlap (mainly with PT-BR) and data processing scale. Thi…
SEQUOR: A Multi-Turn Benchmark for Realistic Constraint Following
Beatriz Canaverde, Duarte M. Alves, José Pombal +2
In a conversation, a helpful assistant must reliably follow user directives, even as they refine, modify, or contradict earlier requests. Yet most instruction-following benchmarks…
A Recipe for Long-Context Reasoning in Large Language Models via On-Policy Optimization and Distillation
Miguel Moura Ramos, Duarte M. Alves, André F. T. Martins
Existing approaches to post-train models for long-context tasks face complementary limitations: (i) supervised fine-tuning (SFT) provides stable supervision but suffers from exposu…
AMALIA Technical Report: A Fully Open Source Large Language Model for European Portuguese
Afonso Simplício, Gonçalo Vinagre, Miguel Moura Ramos +19
Despite rapid progress in open large language models (LLMs), European Portuguese (pt-PT) remains underrepresented in both training data and native evaluation, with machine-translat…
EuroLLM-22B: Technical Report
Miguel Moura Ramos, Duarte M. Alves, Hippolyte Gisserot-Boukhlef +15
This report presents EuroLLM-22B, a large language model trained from scratch to support the needs of European citizens by covering all 24 official European Union languages and 11…
Should We Still Pretrain Encoders with Masked Language Modeling?
Hippolyte Gisserot-Boukhlef, Nicolas Boizard, Manuel Faysse +5
Learning high-quality text representations is fundamental to a wide range of NLP tasks. While encoder pretraining has traditionally relied on Masked Language Modeling (MLM), recent…
EuroLLM-9B: Technical Report
Pedro Henrique Martins, João Alves, Patrick Fernandes +14
This report presents EuroLLM-9B, a large language model trained from scratch to support the needs of European citizens by covering all 24 official European Union languages and 11 a…
EuroBERT: Scaling Multilingual Encoders for European Languages
Nicolas Boizard, Hippolyte Gisserot-Boukhlef, Duarte M. Alves +16
General-purpose multilingual vector representations, used in retrieval, regression and classification, are traditionally obtained from bidirectional encoder models. Despite their w…
EuroLLM: Multilingual Language Models for Europe
Pedro Henrique Martins, Patrick Fernandes, João Alves +12
The quality of open-weight LLMs has seen significant improvement, yet they remain predominantly focused on English. In this paper, we introduce the EuroLLM project, aimed at develo…
Tower: An Open Multilingual Large Language Model for Translation-Related Tasks
Duarte M. Alves, José Pombal, Nuno M. Guerreiro +10
While general-purpose large language models (LLMs) demonstrate proficiency on multiple tasks within the domain of translation, approaches based on open LLMs are competitive only wh…
CroissantLLM: A Truly Bilingual French-English Language Model
Manuel Faysse, Patrick Fernandes, Nuno M. Guerreiro +13
We introduce CroissantLLM, a 1.3B language model pretrained on a set of 3T English and French tokens, to bring to the research and industrial community a high-performance, fully op…
Steering Large Language Models for Machine Translation with Finetuning and In-Context Learning
Duarte M. Alves, Nuno M. Guerreiro, João Alves +5
Large language models (LLMs) are a promising avenue for machine translation (MT). However, current LLM-based MT systems are brittle: their effectiveness highly depends on the choic…
Hallucinations in Large Multilingual Translation Models
Nuno M. Guerreiro, Duarte Alves, Jonas Waldendorf +4
Large-scale multilingual machine translation systems have demonstrated remarkable ability to translate directly between numerous languages, making them increasingly appealing for r…
Turning the Tables: Biased, Imbalanced, Dynamic Tabular Datasets for ML Evaluation
Sérgio Jesus, José Pombal, Duarte Alves +5
Evaluating new techniques on realistic datasets plays a crucial role in the development of ML research and its broader adoption by practitioners. In recent years, there has been a…
CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Task
Ricardo Rei, Marcos Treviso, Nuno M. Guerreiro +9
We present the joint contribution of IST and Unbabel to the WMT 2022 Shared Task on Quality Estimation (QE). Our team participated on all three subtasks: (i) Sentence and Word-leve…