6 papers
Mult-DPO: Multinomial Direct Preference Optimization for Recommender Systems
Yaochen Zhu, Harald Steck, James McInerney +4
Direct preference optimization (DPO) is a simple and effective alignment strategy for large language models (LLMs) based on pairwise preferences. In recommender systems, however, u…
Entropy After </Think> for reasoning model early exiting
Xi Wang, James McInerney, Lequn Wang +1
Reasoning LLMs show improved performance with longer chains of thought. However, recent work has highlighted their tendency to overthink, continuing to revise answers even after re…
Optimization of Epsilon-Greedy Exploration
Ethan Che, Hakan Ceylan, James McInerney +1
Modern recommendation systems rely on exploration to learn user preferences for new items, typically implementing uniform exploration policies (e.g., epsilon-greedy) due to their s…
Adjusting Regression Models for Conditional Uncertainty Calibration
Ruijiang Gao, Mingzhang Yin, James McInerney +1
Conformal Prediction methods have finite-sample distribution-free marginal coverage guarantees. However, they generally do not offer conditional coverage guarantees, which can be i…
Variation Due to Regularization Tractably Recovers Bayesian Deep Learning
James McInerney, Nathan Kallus
Uncertainty quantification in deep learning is crucial for safe and reliable decision-making in downstream tasks. Existing methods quantify uncertainty at the last layer or other a…
Switching the Loss Reduces the Cost in Batch (Offline) Reinforcement Learning
Alex Ayoub, Kaiwen Wang, Vincent Liu +5
We propose training fitted Q-iteration with log-loss (FQI-log) for batch reinforcement learning (RL). We show that the number of samples needed to learn a near-optimal policy with…