13 citations · 24 across the 15 of their papers we have counts for
17 papers
EPIG-Tree: Compute-Optimal Branching for Gradient-Efficient Reinforcement Learning
Nikita Khomich, Leopold Hermansson, Ido Hakimi
Reward-based reinforcement learning for language models, exemplified by Group Relative Policy Optimization (GRPO), collapses an entire stochastic trajectory into a single scalar re…
Majority Voting for Code Generation
Tim Launer, Jonas Hübotter, Marco Bagatella +2
We investigate Functional Majority Voting (FMV), a method based on functional consensus for code generation with Large Language Models, which identifies a representative solution f…
ActiveUltraFeedback: Efficient Preference Data Generation using Active Learning
Davit Melikidze, Marian Schneider, Jessica Lam +4
Reinforcement Learning from Human Feedback (RLHF) has become the standard for aligning Large Language Models (LLMs), yet its efficacy is bottlenecked by the high cost of acquiring…
RewardUQ: A Unified Framework for Uncertainty-Aware Reward Models
Daniel Yang, Samuel Stante, Florian Redhardt +5
Reward models are central to aligning large language models (LLMs) with human preferences. Yet most approaches rely on pointwise reward estimates that overlook the epistemic uncert…
Reinforcement Learning via Self-Distillation
Jonas Hübotter, Frederike Lübeck, Lejs Behric +8
Large language models are increasingly post-trained with reinforcement learning in verifiable domains such as code and math. Yet, current methods for reinforcement learning with ve…
Learning on the Job: Test-Time Curricula for Targeted Reinforcement Learning
Jonas Hübotter, Leander Diaz-Bone, Ido Hakimi +2
Humans are good at learning on the job: We learn how to solve the tasks we face as we go along. Can a model do the same? We propose an agent that assembles a task-specific curricul…
Apertus: Democratizing Open and Compliant LLMs for Global Language Environments
Project Apertus, Alejandro Hernández-Cano, Alexander Hägele +100
We present Apertus, a fully open suite of large language models (LLMs) designed to address two systemic shortcomings in today's open model ecosystem: data compliance and multilingu…
Maximizing Prefix-Confidence at Test-Time Efficiently Improves Mathematical Reasoning
Matthias Otth, Jonas Hübotter, Ido Hakimi +1
Recent work has shown that language models can self-improve by maximizing their own confidence in their predictions, without relying on external verifiers or reward signals. In thi…
From Problem-Solving to Teaching Problem-Solving: Aligning LLMs with Pedagogy using Reinforcement Learning
David Dinucu-Jianu, Jakub Macina, Nico Daheim +3
Large language models (LLMs) can transform education, but their optimization for direct question-answering often undermines effective pedagogy which requires strategically withhold…
Local Mixtures of Experts: Essentially Free Test-Time Training via Model Merging
Ryo Bertolissi, Jonas Hübotter, Ido Hakimi +1
Mixture of expert (MoE) models are a promising approach to increasing model capacity without increasing inference cost, and are core components of many state-of-the-art language mo…
MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutors
Jakub Macina, Nico Daheim, Ido Hakimi +3
Evaluating the pedagogical capabilities of AI-based tutoring models is critical for making guided progress in the field. Yet, we lack a reliable, easy-to-use, and simple-to-run eva…
Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs
Jonas Hübotter, Sascha Bongni, Ido Hakimi +1
Recent efforts in fine-tuning language models often rely on automatic data selection, commonly using Nearest Neighbors retrieval from large datasets. However, we theoretically show…
q2d: Turning Questions into Dialogs to Teach Models How to Search
Yonatan Bitton, Shlomi Cohen-Ganor, Ido Hakimi +3
One of the exciting capabilities of recent language models for dialog is their ability to independently search for relevant information to ground a given dialog response. However,…
Learning Under Delayed Feedback: Implicitly Adapting to Gradient Delays
Rotem Zamir Aviv, Ido Hakimi, Assaf Schuster +1
We consider stochastic convex optimization problems, where several machines act asynchronously in parallel while sharing a common memory. We propose a robust training method for th…
Gap Aware Mitigation of Gradient Staleness
Saar Barkai, Ido Hakimi, Assaf Schuster
Cloud computing is becoming increasingly popular as a platform for distributed training of deep neural networks. Synchronous stochastic gradient descent (SSGD) suffers from substan…
Taming Momentum in a Distributed Asynchronous Environment
Ido Hakimi, Saar Barkai, Moshe Gabel +1
Although distributed computing can significantly reduce the training time of deep neural networks, scaling the training process while maintaining high efficiency and final accuracy…
Faster Neural Network Training with Approximate Tensor Operations
Menachem Adelman, Kfir Y. Levy, Ido Hakimi +1
We propose a novel technique for faster deep neural network training which systematically applies sample-based approximation to the constituent tensor operations, i.e., matrix mult…