6 papers
Monotone and Conservative Policy Iteration Beyond the Tabular Case
S. R. Eshwar, Gugan Thoppe, Ananyabrata Barua +2
We introduce Reliable Policy Iteration (RPI) and Conservative RPI (CRPI), variants of Policy Iteration (PI) and Conservative PI (CPI), that retain tabular guarantees under function…
More Test-Time Compute Can Hurt: Overestimation Bias in LLM Beam Search
Gal Dalal, Assaf Hallak, Gal Chechik +1
Wider beam search should improve LLM reasoning, but when should you stop widening? Prior work on beam width selection has focused on inference efficiency \citep{qin2025dsbd, freita…
Reliable Policy Iteration: Performance Robustness Across Architecture and Environment Perturbations
S. R. Eshwar, Aniruddha Mukherjee, Kintan Saha +4
In a recent work, we proposed Reliable Policy Iteration (RPI), that restores policy iteration's monotonicity-of-value-estimates property to the function approximation setting. Here…
Reinforcement Learning with Segment Feedback
Yihan Du, Anna Winnicki, Gal Dalal +2
Standard reinforcement learning (RL) assumes that an agent can observe a reward for each state-action pair. However, in practical applications, it is often difficult and costly to…
Gradient Boosting Reinforcement Learning
Benjamin Fuhrer, Chen Tessler, Gal Dalal
We present Gradient Boosting Reinforcement Learning (GBRL), a framework that adapts the strengths of gradient boosting trees (GBT) to reinforcement learning (RL) tasks. While neura…
Policy Gradient with Tree Expansion
Gal Dalal, Assaf Hallak, Gugan Thoppe +2
Policy gradient methods are notorious for having a large variance and high sample complexity. To mitigate this, we introduce SoftTreeMax -- a generalization of softmax that employs…