19 citations · 24 across the 19 of their papers we have counts for
7 papers · 1 filter
Off-Policy Learning to Reason Works Because It Is More Pessimistic Than You Think
Otmane Sakhi, Aleksei Arzhantsev, Imad Aouali +1
Large scale reinforcement learning has become a central tool for improving reasoning in large language models. At this scale, generation is often lagged or asynchronous, so updates…
Self-Consistency via Marginal Sharpening
Aleksei Arzhantsev, Otmane Sakhi, Nicolas Chopin
Inference-time sampling can elicit strong reasoning abilities from language models without additional training. Existing power-sampling methods do so by sharpening the distribution…
Learning to Bid in Repeated Second-Price Auctions with Dynamic Values and Aggregated Feedback
Benjamin Heymann, Otmane Sakhi
We study the problem of learning to bid when the bidder's value is dynamic, i.e., when the current value depends on past outcomes. Specifically, we consider a bidder participating…
Shapley-based Data Valuation for LLM Alignment via Sequential Preference Optimization
Mélissa Tamine, Otmane Sakhi, Benjamin Heymann +2
Data valuation is a natural framework for understanding which data sources matter most when aligning a Large Language Model (LLM) from multiple sources. The standard game-theoretic…
RoiRL: Efficient, Self-Supervised Reasoning with Offline Iterative Reinforcement Learning
Aleksei Arzhantsev, Otmane Sakhi, Flavian Vasile
Reinforcement learning (RL) is central to improving reasoning in large language models (LLMs) but typically requires ground-truth rewards. Test-Time Reinforcement Learning (TTRL) r…
Offline Contextual Bandit with Counterfactual Sample Identification
Alexandre Gilotte, Otmane Sakhi, Imad Aouali +1
In production systems, contextual bandit approaches often rely on direct reward models that take both action and context as input. However, these models can suffer from confounding…
Fast Slate Policy Optimization: Going Beyond Plackett-Luce
Otmane Sakhi, David Rohde, Nicolas Chopin
An increasingly important building block of large scale machine learning systems is based on returning slates; an ordered lists of items given a query. Applications of this technol…