9 papers
GRPO Beyond English: A Large-Scale Study of GRPO in Non-English and Multilingual Settings
Konstantin Dobler, Federico Scozzafava, Jonathan Janke +2
Reinforcement Learning with Verifiable Rewards (RLVR), often optimized with Group Relative Policy Optimization (GRPO), has become a central recipe for improving the reasoning capab…
mAceReason-Math: A Dataset of High-Quality Multilingual Math Problems Ready For RLVR
Konstantin Dobler, Simon Lehnerer, Federico Scozzafava +2
Reinforcement Learning with Verifiable Rewards (RLVR) has been successfully applied to significantly boost the capabilities of pretrained large language models, especially in the m…
Multilingual Reasoning Gym: Multilingual Scaling of Procedural Reasoning Environments
Konstantin Dobler, Simon Lehnerer, Federico Scozzafava +2
We present the Multilingual Reasoning Gym, an extension of Reasoning Gym (Stojanovski et al., 2025), that procedurally generates verifiable reasoning problems across 14 languages.…
CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data
Pedro Ortiz Suarez, Laurie Burchell, Catherine Arnett +94
Language identification (LID) is a fundamental step in curating multilingual corpora. However, LID models still perform poorly for many languages, especially on the noisy and heter…
Token Distillation: Attention-aware Input Embeddings For New Tokens
Konstantin Dobler, Desmond Elliott, Gerard de Melo
Current language models rely on static vocabularies determined at pretraining time, which can lead to decreased performance and increased computational cost for domains underrepres…
I Don't Know: Explicit Modeling of Uncertainty with an [IDK] Token
Roi Cohen, Konstantin Dobler, Eden Biran +1
Large Language Models are known to capture real-world knowledge, allowing them to excel in many downstream tasks. Despite recent advances, these models are still prone to what are…
Language Adaptation on a Tight Academic Compute Budget: Tokenizer Swapping Works and Pure bfloat16 Is Enough
Konstantin Dobler, Gerard de Melo
We investigate continued pretraining of LLMs for language adaptation on a tight academic budget: a setting in which only a few GPUs can be used in parallel, for a heavily constrain…
Efficient Parallelization Layouts for Large-Scale Distributed Model Training
Johannes Hagemann, Samuel Weinbach, Konstantin Dobler +2
Efficiently training large language models requires parallelizing across hundreds of hardware accelerators and invoking various compute and memory optimizations. When combined, man…
Art Creation with Multi-Conditional StyleGANs
Konstantin Dobler, Florian Hübscher, Jan Westphal +3
Creating meaningful art is often viewed as a uniquely human endeavor. A human artist needs a combination of unique skills, understanding, and genuine intention to create artworks t…