2 citations · 5 across the 17 of their papers we have counts for
16 papers · 1 filter
EuroLLM-22B: Technical Report
Miguel Moura Ramos, Duarte M. Alves, Hippolyte Gisserot-Boukhlef +15
This report presents EuroLLM-22B, a large language model trained from scratch to support the needs of European citizens by covering all 24 official European Union languages and 11…
Multilingual Contextualization of Large Language Models for Document-Level Machine Translation
Miguel Moura Ramos, Patrick Fernandes, Sweta Agrawal +1
Large language models (LLMs) have demonstrated strong performance in sentence-level machine translation, but scaling to document-level translation remains challenging, particularly…
Do LLMs Understand Your Translations? Evaluating Paragraph-level MT with Question Answering
Patrick Fernandes, Sweta Agrawal, Emmanouil Zaranis +2
Despite the steady progress in machine translation evaluation, existing automatic metrics struggle to capture how well meaning is preserved beyond sentence boundaries. We posit tha…
M-Prometheus: A Suite of Open Multilingual LLM Judges
José Pombal, Dongkeun Yoon, Patrick Fernandes +5
The use of language models for automatically evaluating long-form text (LLM-as-a-judge) is becoming increasingly common, yet most LLM judges are optimized exclusively for English,…
Translate Smart, not Hard: Cascaded Translation Systems with Quality-Aware Deferral
António Farinhas, Nuno M. Guerreiro, Sweta Agrawal +2
Larger models often outperform smaller ones but come with high computational costs. Cascading offers a potential solution. By default, it uses smaller models and defers only some i…
Fine-Grained Reward Optimization for Machine Translation using Error Severity Mappings
Miguel Moura Ramos, Tomás Almeida, Daniel Vareta +4
Reinforcement learning (RL) has been proven to be an effective and robust method for training neural machine translation systems, especially when paired with powerful reward models…
Modeling User Preferences with Automatic Metrics: Creating a High-Quality Preference Dataset for Machine Translation
Sweta Agrawal, José G. C. de Souza, Ricardo Rei +5
Alignment with human preferences is an important step in developing accurate and safe large language models. This is no exception in machine translation (MT), where better handling…
Watching the Watchers: Exposing Gender Disparities in Machine Translation Quality Estimation
Emmanouil Zaranis, Giuseppe Attanasio, Sweta Agrawal +1
Quality estimation (QE)-the automatic assessment of translation quality-has recently become crucial across several stages of the translation pipeline, from data curation to trainin…
Reranking Laws for Language Generation: A Communication-Theoretic Perspective
António Farinhas, Haau-Sing Li, André F. T. Martins
To ensure large language models (LLMs) are used safely, one must reduce their propensity to hallucinate or to generate unacceptable answers. A simple and often used strategy is to…
DOCE: Finding the Sweet Spot for Execution-Based Code Generation
Haau-Sing Li, Patrick Fernandes, Iryna Gurevych +1
Recently, a diverse set of decoding and reranking procedures have been shown effective for LLM-based code generation. However, a comprehensive framework that links and experimental…
QUEST: Quality-Aware Metropolis-Hastings Sampling for Machine Translation
Gonçalo R. A. Faria, Sweta Agrawal, António Farinhas +3
An important challenge in machine translation (MT) is to generate high-quality and diverse translations. Prior work has shown that the estimated likelihood from the MT model correl…
Conformal Prediction for Natural Language Processing: A Survey
Margarida M. Campos, António Farinhas, Chrysoula Zerva +2
The rapid proliferation of large language models and natural language processing (NLP) applications creates a crucial need for uncertainty quantification to mitigate risks such as…
Can Automatic Metrics Assess High-Quality Translations?
Sweta Agrawal, António Farinhas, Ricardo Rei +1
Automatic metrics for evaluating translation quality are typically validated by measuring how well they correlate with human assessments. However, correlation methods tend to captu…
Non-Exchangeable Conformal Language Generation with Nearest Neighbors
Dennis Ulmer, Chrysoula Zerva, André F. T. Martins
Quantifying uncertainty in automatically generated text is important for letting humans check potential hallucinations and making systems more reliable. Conformal prediction is an…
Aligning Neural Machine Translation Models: Human Feedback in Training and Inference
Miguel Moura Ramos, Patrick Fernandes, António Farinhas +1
Reinforcement learning from human feedback (RLHF) is a recent technique to improve the quality of the text generated by a language model, making it closer to what humans would gene…
An Empirical Study of Translation Hypothesis Ensembling with Large Language Models
António Farinhas, José G. C. de Souza, André F. T. Martins
Large language models (LLMs) are becoming a one-fits-many solution, but they sometimes hallucinate or produce unreliable output. In this paper, we investigate how hypothesis ensemb…