8 papers
Revisiting Zeroth-Order Hessian Approximation: A Single-Step Policy Optimization Lens
Junbin Qiu, Zhaowei Hong, Renzhe Xu +1
Accurate Zeroth-Order (ZO) Hessian estimation is a cornerstone of derivative-free methods, essential for tasks such as bilevel optimization, Bayesian inference, and uncertainty qua…
Reference-Sampled Boltzmann Projection for KL-Regularized RLVR: Target-Matched Weighted SFT, Finite One-Shot Gaps, and Policy Mirror Descent
Yao Shu, Chenxing Wei, Hongbin Lin +2
Online reinforcement learning with verifiable rewards (RLVR) turns checkable outcomes into a scalable training signal, but it keeps rollout generation, verifier scoring, and refere…
Scheduling Your LLM Reinforcement Learning with Reasoning Trees
Hong Wang, Zhezheng Hao, Jian Luo +6
Using Reinforcement Learning with Verifiable Rewards (RLVR) to optimize Large Language Models (LLMs) can be conceptualized as progressively editing a query's `Reasoning Tree'. This…
Model-based Offline RL via Robust Value-Aware Model Learning with Implicitly Differentiable Adaptive Weighting
Zhongjian Qiao, Jiafei Lyu, Boxiang Lyu +3
Model-based offline reinforcement learning (RL) aims to enhance offline RL with a dynamics model that facilitates policy exploration. However, \textit{model exploitation} could occ…
LFPO: Likelihood-Free Policy Optimization for Masked Diffusion Models
Chenxing Wei, Jiazhen Kang, Hong Wang +8
Reinforcement Learning with Verifiable Rewards (RLVR) has achieved remarkable success in improving autoregressive models, especially in domains requiring correctness like mathemati…
Words & Weights: Streamlining Multi-Turn Interactions via Co-Adaptation
Chenxing Wei, Hong Wang, Ying He +4
Test-time policy adaptation for multi-turn interactions (T2PAM) is essential for aligning Large Language Models (LLMs) with dynamic user needs during inference time. However, exist…