19 citations · 24 across the 19 of their papers we have counts for
20 papers
Off-Policy Learning to Reason Works Because It Is More Pessimistic Than You Think
Otmane Sakhi, Aleksei Arzhantsev, Imad Aouali +1
Large scale reinforcement learning has become a central tool for improving reasoning in large language models. At this scale, generation is often lagged or asynchronous, so updates…
Self-Consistency via Marginal Sharpening
Aleksei Arzhantsev, Otmane Sakhi, Nicolas Chopin
Inference-time sampling can elicit strong reasoning abilities from language models without additional training. Existing power-sampling methods do so by sharpening the distribution…
Learning to Bid in Repeated Second-Price Auctions with Dynamic Values and Aggregated Feedback
Benjamin Heymann, Otmane Sakhi
We study the problem of learning to bid when the bidder's value is dynamic, i.e., when the current value depends on past outcomes. Specifically, we consider a bidder participating…
RecoAtlas: From Semantic Plausibility to Set-Level Utility in LLM Recommendation Agents
Imad Aouali, Flavian Vasile, Otmane Sakhi +2
LLM recommendation agents increasingly produce structured recommendation reports: sets of items accompanied by natural-language justifications. Yet existing evaluations often reduc…
Shapley-based Data Valuation for LLM Alignment via Sequential Preference Optimization
Mélissa Tamine, Otmane Sakhi, Benjamin Heymann +2
Data valuation is a natural framework for understanding which data sources matter most when aligning a Large Language Model (LLM) from multiple sources. The standard game-theoretic…
RoiRL: Efficient, Self-Supervised Reasoning with Offline Iterative Reinforcement Learning
Aleksei Arzhantsev, Otmane Sakhi, Flavian Vasile
Reinforcement learning (RL) is central to improving reasoning in large language models (LLMs) but typically requires ground-truth rewards. Test-Time Reinforcement Learning (TTRL) r…
Offline Contextual Bandit with Counterfactual Sample Identification
Alexandre Gilotte, Otmane Sakhi, Imad Aouali +1
In production systems, contextual bandit approaches often rely on direct reward models that take both action and context as input. However, these models can suffer from confounding…
Non-Linear Counterfactual Aggregate Optimization
Benjamin Heymann, Otmane Sakhi
We consider the problem of directly optimizing a non-linear function of an outcome, where this outcome itself is the sum of many small contributions. The non-linearity of the funct…
Off-Policy Learning in Large Action Spaces: Optimization Matters More Than Estimation
Imad Aouali, Otmane Sakhi
Off-policy evaluation (OPE) and off-policy learning (OPL) are foundational for decision-making in offline contextual bandits. Recent advances in OPL primarily optimize OPE estimato…
Exploiting Similarities in A/B Testing with Off-Policy Estimation
Otmane Sakhi, Alexandre Gilotte, David Rohde
We study A/B testing, the standard protocol for measuring the performance gain of a new decision system relative to a baseline. Traditional A/B testing treats both systems as black…
Sequential Off-Policy Learning with Logarithmic Smoothing
Maxime Haddouche, Otmane Sakhi
Off-policy learning enables training policies from logged interaction data. Most prior work considers the batch setting, where a policy is learned from data generated by a single b…
Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learning
Otmane Sakhi, Imad Aouali, Pierre Alquier +1
This work investigates the offline formulation of the contextual bandit problem, where the goal is to leverage past interactions collected under a behavior policy to evaluate, sele…
Fast Slate Policy Optimization: Going Beyond Plackett-Luce
Otmane Sakhi, David Rohde, Nicolas Chopin
An increasingly important building block of large scale machine learning systems is based on returning slates; an ordered lists of items given a query. Applications of this technol…
PAC-Bayesian Offline Contextual Bandits With Guarantees
Otmane Sakhi, Pierre Alquier, Nicolas Chopin
This paper introduces a new principled approach for off-policy learning in contextual bandits. Unlike previous work, our approach does not derive learning principles from intractab…
Offline Evaluation of Reward-Optimizing Recommender Systems: The Case of Simulation
Imad Aouali, Amine Benhalloum, Martin Bompaire +5
Both in academic and industry-based research, online evaluation methods are seen as the golden standard for interactive applications like recommendation systems. Naturally, the rea…
Probabilistic Rank and Reward: A Scalable Model for Slate Recommendation
Imad Aouali, Achraf Ait Sidi Hammou, Otmane Sakhi +2
We introduce Probabilistic Rank and Reward (PRR), a scalable probabilistic model for personalized slate recommendation. Our approach allows off-policy estimation of the reward in t…
Fast Offline Policy Optimization for Large Scale Recommendation
Otmane Sakhi, David Rohde, Alexandre Gilotte
Personalised interactive systems such as recommender systems require selecting relevant items from massive catalogs dependent on context. Reward-driven offline optimisation of thes…
Improving Offline Contextual Bandits with Distributional Robustness
Otmane Sakhi, Louis Faury, Flavian Vasile
This paper extends the Distributionally Robust Optimization (DRO) approach for offline contextual bandits. Specifically, we leverage this framework to introduce a convex reformulat…
BLOB : A Probabilistic Model for Recommendation that Combines Organic and Bandit Signals
Otmane Sakhi, Stephen Bonner, David Rohde +1
A common task for recommender systems is to build a pro le of the interests of a user from items in their browsing history and later to recommend items to the user from the same ca…
Reconsidering Analytical Variational Bounds for Output Layers of Deep Networks
Otmane Sakhi, Stephen Bonner, David Rohde +1
The combination of the re-parameterization trick with the use of variational auto-encoders has caused a sensation in Bayesian deep learning, allowing the training of realistic gene…