12 papers
When Refusal Looks Safe: The Refusal-Cue Shortcut in Safety Guard Models
Yu Feng, Chunting Zang, Chen Shen +4
Safety guards are widely used to filter harmful content and are typically trained via supervised fine-tuning on labeled prompt-response pairs. We audit two widely used safety-guard…
Purified OPSD: On-Policy Self-Distillation Without Losing How to Think
Zhanming Shen, Jintao Tong, Shaotian Yan +9
On-policy self-distillation (OPSD) has emerged as a promising paradigm for improving LLM reasoning, where a privileged teacher with access to reference solutions provides token-lev…
OPPO: Bayesian Value Recursion for Token-Level Credit Assignment in LLM Reasoning
Yu Li, Rui Miao, Tian Lan +1
Reinforcement learning with verifiable rewards has become the standard recipe for improving LLM reasoning, but the dominant algorithm GRPO assigns a single trajectory-level advanta…
Backtracking When It Strays: Mitigating Dual Exposure Biases in LLM Reasoning Distillation
Bing Wang, Shaotian Yan, Chen Shen +7
Large language models (LLMs) have achieved remarkable success in complex reasoning tasks via long chain-of-thought (CoT), yet their immense computational overhead hinders real-worl…
Are Rationales Necessary and Sufficient? Tuning LLMs for Explainable Misinformation Detection
Bing Wang, Rui Miao, Ximing Li +6
The rapid spread of misinformation on social media platforms has become a formidable challenge. To mitigate its proliferation, Misinformation Detection (MD) has emerged as a critic…
TOPPO: Rethinking PPO for Multi-Task Reinforcement Learning with Critic Balancing
Yuanpeng Li, Gefei Lin, Annie Qu +1
Soft Actor-Critic (SAC) and its variants dominate Multi-Task Reinforcement Learning (MTRL) due to their off-policy sample efficiency, while on-policy methods such as Proximal Polic…