2 citations · 8 across the 12 of their papers we have counts for
13 papers
Value-Guided Search for Efficient Chain-of-Thought Reasoning
Kaiwen Wang, Jin Peng Zhou, Jonathan Chang +4
In this paper, we propose a simple and efficient method for value model training on long-context reasoning traces. Compared to existing process reward models (PRMs), our method doe…
: Provably Optimal Distributional RL for LLM Post-Training
Jin Peng Zhou, Kaiwen Wang, Jonathan Chang +5
Reinforcement learning (RL) post-training is crucial for LLM alignment and reasoning, but existing policy-based methods, such as PPO and DPO, can fall short of fixing shortcuts inh…
The Central Role of the Loss Function in Reinforcement Learning
Kaiwen Wang, Nathan Kallus, Wen Sun
This paper illustrates the central role of loss functions in data-driven decision making, providing a comprehensive survey on their influence in cost-sensitive classification (CSC)…
Efficient and Sharp Off-Policy Evaluation in Robust Markov Decision Processes
Andrew Bennett, Nathan Kallus, Miruna Oprescu +2
We study the evaluation of a policy under best- and worst-case perturbations to a Markov decision process (MDP), using transition observations from the original MDP, whether they a…
A Reductions Approach to Risk-Sensitive Reinforcement Learning with Optimized Certainty Equivalents
Kaiwen Wang, Dawen Liang, Nathan Kallus +1
We study risk-sensitive RL where the goal is learn a history-dependent policy that optimizes some risk measure of cumulative rewards. We consider a family of risks called the optim…
Switching the Loss Reduces the Cost in Batch (Offline) Reinforcement Learning
Alex Ayoub, Kaiwen Wang, Vincent Liu +5
We propose training fitted Q-iteration with log-loss (FQI-log) for batch reinforcement learning (RL). We show that the number of samples needed to learn a near-optimal policy with…
More Benefits of Being Distributional: Second-Order Bounds for Reinforcement Learning
Kaiwen Wang, Owen Oertell, Alekh Agarwal +2
In this paper, we prove that Distributional Reinforcement Learning (DistRL), which learns the return distribution, can obtain second-order bounds in both online and offline RL in g…
JoinGym: An Efficient Query Optimization Environment for Reinforcement Learning
Kaiwen Wang, Junxiong Wang, Yueying Li +3
Join order selection (JOS) is the problem of ordering join operations to minimize total query execution cost and it is the core NP-hard combinatorial optimization problem of query…
The Benefits of Being Distributional: Small-Loss Bounds for Reinforcement Learning
Kaiwen Wang, Kevin Zhou, Runzhe Wu +2
While distributional reinforcement learning (DistRL) has been empirically effective, the question of when and why it is better than vanilla, non-distributional RL has remained unan…
Near-Minimax-Optimal Risk-Sensitive Reinforcement Learning with CVaR
Kaiwen Wang, Nathan Kallus, Wen Sun
In this paper, we study risk-sensitive Reinforcement Learning (RL), focusing on the objective of Conditional Value at Risk (CVaR) with risk tolerance . Starting with multi-arm b…
Learning Bellman Complete Representations for Offline Policy Evaluation
Jonathan D. Chang, Kaiwen Wang, Nathan Kallus +1
We study representation learning for Offline Reinforcement Learning (RL), focusing on the important task of Offline Policy Evaluation (OPE). Recent work shows that, in contrast to…
Provable Benefits of Representational Transfer in Reinforcement Learning
Alekh Agarwal, Yuda Song, Wen Sun +3
We study the problem of representational transfer in RL, where an agent first pretrains in a number of source tasks to discover a shared representation, which is subsequently used…
Doubly Robust Distributionally Robust Off-Policy Evaluation and Learning
Nathan Kallus, Xiaojie Mao, Kaiwen Wang +1
Off-policy evaluation and learning (OPE/L) use offline observational data to make better decisions, which is crucial in applications where online experimentation is limited. Howeve…