13 papers
What Should a Self-Teacher See? Privileged Context Design for On-Policy Self-Distillation
Kanghui Tian, Siyuan Liu, Tianxiang Jiang +8
More privileged information does not always make a better teacher. We study this tension in on-policy self-distillation (OPSD), where a self-teacher scores the student's own rollou…
SafeRI: Recognition and Intervention for Token-Level Safety Intervention in Large Vision Language Models
Caoyuan Ma, Tian Gu, Wenpu Liu +11
Existing safety alignment methods for vision-language models usually modify the model behavior globally: once the safety parameters are trained or loaded, they participate in both…
SafeCap: Improving LVLM Safety with Image Captioning Reinforcement Learning
Caoyuan Ma, Wenpu Liu, Weichu Xie +12
Large vision-language models (LVLMs) remain vulnerable to jailbreak attacks that exploit visual inputs to bypass safety alignment inherited from their language backbones. We propos…
OSEF: One-Step Evidence Fusion for Cross-Video Scene Procedure Planning
Zhentong Ye, Lei Zhang, Sijia Zhou +7
Video Scene Procedure Planning (VSPP) supplies the target start-goal observations in advance, leaving open how a planner should act when the evidence must itself be retrieved. We i…
DOPD: Dual On-policy Distillation
Xinlei Yu, Gen Li, Qingyi Si +13
On-policy distillation (OPD) offers superior capacity transfer by supervising student-sampled trajectories with dense token-level signals. To furnish high-quality supervision sourc…
Leveraging Error Diversity in Group Rollouts for Reinforcement Learning
Wenpu Liu, Yuqi Xu, Weichu Xie +8
Reinforcement Learning from Verifiable Rewards (RLVR) typically samples multiple responses per prompt and assigns binary rewards based on individual correctness, yet the collective…
ViCuR: Visual Cues as Recoverable Privilege for Multimodal On-Policy Distillation
Kanghui Tian, Siyuan Liu, Ziang Yan +3
On-policy distillation (OPD) improves reasoning by training a student on trajectories sampled from its own policy under supervision from a teacher. In multimodal reasoning, a commo…
Right Makes Might: Aligning Verified Hidden States Empowers RL Reasoning
Ziyue Wang, Aomufei Yuan, Yongfu Zhu +10
Reinforcement Learning from Verifiable Rewards (RLVR) has become the dominant approach for improving mathematical reasoning in large language models, yet current methods reduce eac…
AdaCodec: A Predictive Visual Code for Video MLLMs
Haowen Hou, Zhen Huang, Zheming Liang +8
Video is temporally redundant: adjacent frames usually share most objects, background, and layout. Yet existing video multimodal large language models (video MLLMs) usually encode…
Beyond Visual Memory: Mechanistic Diagnostics of Latent Visual Reasoning
Jiawei Guo, Yu Chen, Xiang Wang +6
Recent latent visual reasoning methods achieve substantial gains by inserting continuous latent tokens into multimodal language models. These gains are commonly attributed to the t…
Step-wise Rubric Rewards for LLM Reasoning
Weichu Xie, Haozhe Zhao, Wenpu Liu +15
Reinforcement Learning with Verifiable Rewards (RLVR) is widely used to improve reasoning in large language models, but rewards only final-answer correctness with no supervision ov…
Interleaved Latent Visual Reasoning with Selective Perceptual Modeling
Shuai Dong, Siyuan Wang, Xingyu Liu +3
Interleaved reasoning paradigms enhance Multimodal Large Language Models (MLLMs) with visual feedback but are hindered by the prohibitive computational cost of re-encoding pixel-de…
Enhancing Reasoning through Process Supervision with Monte Carlo Tree Search
Shuangtao Li, Shuaihao Dong, Kexin Luan +2
Large language models (LLMs) have demonstrated their remarkable capacity across a variety of tasks. However, reasoning remains a challenge for LLMs. To improve LLMs' reasoning abil…