2 citations · 2 across the 24 of their papers we have counts for
11 papers · 1 filter
SwissGov-RSD: A Human-annotated, Cross-lingual Benchmark for Token-level Recognition of Semantic Differences Between Related Documents
Michelle Wastl, Jannis Vamvas, Rico Sennrich
Recognizing semantic differences across documents is crucial for text generation evaluation and content alignment, especially in cross-lingual settings. However, as a standalone ta…
QueST: Incentivizing LLMs to Generate Difficult Problems
Hanxu Hu, Xingxing Zhang, Jannis Vamvas +2
Large Language Models have achieved strong performance on reasoning tasks, solving competition-level coding and math problems. However, their scalability is limited by human-labele…
Measuring the Effect of Disfluency in Multilingual Knowledge Probing Benchmarks
Kirill Semenov, Rico Sennrich
For multilingual factual knowledge assessment of LLMs, benchmarks such as MLAMA use template translations that do not take into account the grammatical and semantic information of…
Expanding the WMT24++ Benchmark with Rumantsch Grischun, Sursilvan, Sutsilvan, Surmiran, Puter, and Vallader
Jannis Vamvas, Ignacio Pérez Prat, Not Battesta Soliva +14
The Romansh language, spoken in Switzerland, has limited resources for machine translation evaluation. In this paper, we present a benchmark for six varieties of Romansh: Rumantsch…
Robust Native Language Identification through Agentic Decomposition
Ahmet Yavuz Uluslu, Tannon Kew, Tilia Ellendorff +2
Large language models (LLMs) often achieve high performance in native language identification (NLI) benchmarks by leveraging superficial contextual clues such as names, locations,…
The Mediomatix Corpus: Parallel Data for Romansh Language Varieties via Comparable Schoolbooks
Zachary Hopton, Jannis Vamvas, Andrin Büchler +3
The five idioms (i.e., varieties) of the Romansh language are largely standardized and are taught in the schools of the respective communities in Switzerland. In this paper, we pre…
Leveraging In-Context Learning for Political Bias Testing of LLMs
Patrick Haller, Jannis Vamvas, Rico Sennrich +1
A growing body of work has been querying LLMs with political questions to evaluate their potential biases. However, this probing method has limited stability, making comparisons be…
20min-XD: A Comparable Corpus of Swiss News Articles
Michelle Wastl, Jannis Vamvas, Selena Calleri +1
We present 20min-XD (20 Minuten cross-lingual document-level), a French-German, document-level comparable corpus of news articles, sourced from the Swiss online news outlet 20 Minu…
CHARM: Calibrating Reward Models With Chatbot Arena Scores
Xiao Zhu, Chenmien Tan, Pinzhen Chen +4
Reward models (RMs) play a crucial role in Reinforcement Learning from Human Feedback by serving as proxies for human preferences in aligning large language models. However, they s…
Source-primed Multi-turn Conversation Helps Large Language Models Translate Documents
Hanxu Hu, Jannis Vamvas, Rico Sennrich
LLMs have paved the way for truly simple document-level machine translation, but challenges such as omission errors remain. In this paper, we study a simple method for handling doc…
Examining Multilingual Embedding Models Cross-Lingually Through LLM-Generated Adversarial Examples
Andrianos Michail, Simon Clematide, Rico Sennrich
The evaluation of cross-lingual semantic search models is often limited to existing datasets from tasks such as information retrieval and semantic textual similarity. We introduce…