14 papers
Fetch Me If You Can: A Comprehensive Analysis of Fetch Metadata
Pedro Bernardo, Alex Ploner, Marco Squarcina +2
Effective server-side defenses against cross-site attacks depend on understanding the context in which browser requests are generated. However, servers often lack the information n…
Last Translation Benchmark
Vilém Zouhar, Niyati Bafna, Mukund Choudhary +263
For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. As models get stronger, stan…
Scaling Unsupervised Word Alignment to Documents via Structural Constraints
Michelle Wastl, Jannis Vamvas, Rico Sennrich
Word alignment has traditionally been studied between sentences, but many cross-lingual tasks increasingly require correspondences across full documents. While recent multilingual…
Reinforcement Learning Elicits Contextual Learning of Unseen Language Translation
Hanxu Hu, Zdeněk Šnajdr, Pinzhen Chen +2
Prior work has shown that large language models (LLMs) can translate unseen or low-resource languages by undergoing continued training or even by encoding a grammar book in their c…
DeReason: A Difficulty-Aware Curriculum Improves Decoupled SFT-then-RL Training for General Reasoning
Hanxu Hu, Yuxuan Wang, Maggie Huan +4
Reinforcement learning with Verifiable Rewards (RLVR) has emerged as a powerful paradigm for eliciting reasoning capabilities in large language models, particularly in mathematics…
Translation Asymmetry in LLMs as a Data Augmentation Factor: A Case Study for 6 Romansh Language Varieties
Jannis Vamvas, Ignacio Pérez Prat, Angela Heldstab +3
Recent strategies for low-resource machine translation rely on LLMs to generate synthetic data based on text in higher-resource languages. We revisit this idea for Romansh, a langu…
Expanding the WMT24++ Benchmark with Rumantsch Grischun, Sursilvan, Sutsilvan, Surmiran, Puter, and Vallader
Jannis Vamvas, Ignacio Pérez Prat, Not Battesta Soliva +14
The Romansh language, spoken in Switzerland, has limited resources for machine translation evaluation. In this paper, we present a benchmark for six varieties of Romansh: Rumantsch…
Apertus: Democratizing Open and Compliant LLMs for Global Language Environments
Project Apertus, Alejandro Hernández-Cano, Alexander Hägele +100
We present Apertus, a fully open suite of large language models (LLMs) designed to address two systemic shortcomings in today's open model ecosystem: data compliance and multilingu…
Leveraging In-Context Learning for Political Bias Testing of LLMs
Patrick Haller, Jannis Vamvas, Rico Sennrich +1
A growing body of work has been querying LLMs with political questions to evaluate their potential biases. However, this probing method has limited stability, making comparisons be…
Fine-tuning the SwissBERT Encoder Model for Embedding Sentences and Documents
Juri Grosjean, Jannis Vamvas
Encoder models trained for the embedding of sentences or short documents have proven useful for tasks such as semantic search and topic modeling. In this paper, we present a versio…
Linear-time Minimum Bayes Risk Decoding with Reference Aggregation
Jannis Vamvas, Rico Sennrich
Minimum Bayes Risk (MBR) decoding is a text generation technique that has been shown to improve the quality of machine translations, but is expensive, even if a sampling-based appr…
Modular Adaptation of Multilingual Encoders to Written Swiss German Dialect
Jannis Vamvas, Noëmi Aepli, Rico Sennrich
Creating neural text encoders for written Swiss German is challenging due to a dearth of training data combined with dialectal variation. In this paper, we build on several existin…
Machine Translation Models are Zero-Shot Detectors of Translation Direction
Michelle Wastl, Jannis Vamvas, Rico Sennrich
Detecting the translation direction of parallel text has applications for machine translation training and evaluation, but also has forensic applications such as resolving plagiari…
Trained MT Metrics Learn to Cope with Machine-translated References
Jannis Vamvas, Tobias Domhan, Sony Trenous +2
Neural metrics trained on human evaluations of MT tend to correlate well with human judgments, but their behavior is not fully understood. In this paper, we perform a controlled ex…