11 papers
Shapley-based Data Valuation for LLM Alignment via Sequential Preference Optimization
Mélissa Tamine, Otmane Sakhi, Benjamin Heymann +2
Data valuation is a natural framework for understanding which preference datasets matter most when aligning a Large Language Model (LLM) using multiple sources. The standard game-t…
Off-Policy Learning in Large Action Spaces: Optimization Matters More Than Estimation
Imad Aouali, Otmane Sakhi
Off-policy evaluation (OPE) and off-policy learning (OPL) are foundational for decision-making in offline contextual bandits. Recent advances in OPL primarily optimize OPE estimato…
Exploiting Similarities in A/B Testing with Off-Policy Estimation
Otmane Sakhi, Alexandre Gilotte, David Rohde
We study A/B testing, the standard protocol for measuring the performance gain of a new decision system relative to a baseline. Traditional A/B testing treats both systems as black…
Off-Policy Learning to Reason Works Because It Is More Pessimistic Than You Think
Otmane Sakhi, Aleksei Arzhantsev, Imad Aouali +1
Large scale reinforcement learning has become a central tool for improving reasoning in large language models. At this scale, generation is often lagged or asynchronous, so updates…
Self-Consistency via Marginal Sharpening
Aleksei Arzhantsev, Otmane Sakhi, Nicolas Chopin
Inference-time sampling can elicit strong reasoning abilities from language models without additional training. Existing power-sampling methods do so by sharpening the distribution…
Learning to Bid in Repeated Second-Price Auctions with Dynamic Values and Aggregated Feedback
Benjamin Heymann, Otmane Sakhi
We study the problem of learning to bid when the bidder's value is dynamic, i.e., when the current value depends on past outcomes. Specifically, we consider a bidder participating…