6 papers · 1 filter
GRPO Beyond English: A Large-Scale Study of GRPO in Non-English and Multilingual Settings
Konstantin Dobler, Federico Scozzafava, Jonathan Janke +2
Reinforcement Learning with Verifiable Rewards (RLVR), often optimized with Group Relative Policy Optimization (GRPO), has become a central recipe for improving the reasoning capab…
mAceReason-Math: A Dataset of High-Quality Multilingual Math Problems Ready For RLVR
Konstantin Dobler, Simon Lehnerer, Federico Scozzafava +2
Reinforcement Learning with Verifiable Rewards (RLVR) has been successfully applied to significantly boost the capabilities of pretrained large language models, especially in the m…
Multilingual Reasoning Gym: Multilingual Scaling of Procedural Reasoning Environments
Konstantin Dobler, Simon Lehnerer, Federico Scozzafava +2
We present the Multilingual Reasoning Gym, an extension of Reasoning Gym (Stojanovski et al., 2025), that procedurally generates verifiable reasoning problems across 14 languages.…
CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data
Pedro Ortiz Suarez, Laurie Burchell, Catherine Arnett +94
Language identification (LID) is a fundamental step in curating multilingual corpora. However, LID models still perform poorly for many languages, especially on the noisy and heter…
Token Distillation: Attention-aware Input Embeddings For New Tokens
Konstantin Dobler, Desmond Elliott, Gerard de Melo
Current language models rely on static vocabularies determined at pretraining time, which can lead to decreased performance and increased computational cost for domains underrepres…
Language Adaptation on a Tight Academic Compute Budget: Tokenizer Swapping Works and Pure bfloat16 Is Enough
Konstantin Dobler, Gerard de Melo
We investigate continued pretraining of LLMs for language adaptation on a tight academic budget: a setting in which only a few GPUs can be used in parallel, for a heavily constrain…