12 papers
When Refusal Looks Safe: The Refusal-Cue Shortcut in Safety Guard Models
Yu Feng, Chunting Zang, Chen Shen +4
Safety guards are widely used to filter harmful content and are typically trained via supervised fine-tuning on labeled prompt-response pairs. We audit two widely used safety-guard…
Purified OPSD: On-Policy Self-Distillation Without Losing How to Think
Zhanming Shen, Jintao Tong, Shaotian Yan +9
On-policy self-distillation (OPSD) has emerged as a promising paradigm for improving LLM reasoning, where a privileged teacher with access to reference solutions provides token-lev…
OPPO: Bayesian Value Recursion for Token-Level Credit Assignment in LLM Reasoning
Yu Li, Rui Miao, Tian Lan +1
Reinforcement learning with verifiable rewards has become the standard recipe for improving LLM reasoning, but the dominant algorithm GRPO assigns a single trajectory-level advanta…
Backtracking When It Strays: Mitigating Dual Exposure Biases in LLM Reasoning Distillation
Bing Wang, Shaotian Yan, Chen Shen +7
Large language models (LLMs) have achieved remarkable success in complex reasoning tasks via long chain-of-thought (CoT), yet their immense computational overhead hinders real-worl…
Are Rationales Necessary and Sufficient? Tuning LLMs for Explainable Misinformation Detection
Bing Wang, Rui Miao, Ximing Li +6
The rapid spread of misinformation on social media platforms has become a formidable challenge. To mitigate its proliferation, Misinformation Detection (MD) has emerged as a critic…
TOPPO: Rethinking PPO for Multi-Task Reinforcement Learning with Critic Balancing
Yuanpeng Li, Gefei Lin, Annie Qu +1
Soft Actor-Critic (SAC) and its variants dominate Multi-Task Reinforcement Learning (MTRL) due to their off-policy sample efficiency, while on-policy methods such as Proximal Polic…
On the Step Length Confounding in LLM Reasoning Data Selection
Bing Wang, Rui Miao, Chen Shen +7
Large reasoning models have recently demonstrated strong performance on complex tasks that require long chain-of-thought reasoning, through supervised fine-tuning on large-scale an…
ARISE: Agent Reasoning with Intrinsic Skill Evolution in Hierarchical Reinforcement Learning
Yu Li, Rui Miao, Zhengling Qi +1
The dominant paradigm for improving mathematical reasoning in language models relies on Reinforcement Learning with verifiable rewards. Yet existing methods treat each problem inst…
Explainable and Fine-Grained Safeguarding of LLM Multi-Agent Systems via Bi-Level Graph Anomaly Detection
Junjun Pan, Yixin Liu, Rui Miao +5
Large language model (LLM)-based multi-agent systems (MAS) have shown strong capabilities in solving complex tasks. As MAS become increasingly autonomous in various safety-critical…
Where Did This Sentence Come From? Tracing Provenance in LLM Reasoning Distillation
Kaiyuan Liu, Shaotian Yan, Rui Miao +4
Reasoning distillation has attracted increasing attention. It typically leverages a large teacher model to generate reasoning paths, which are then used to fine-tune a student mode…
Collective Communication for 100k+ GPUs
Min Si, Pavan Balaji, Yongzhou Chen +36
The increasing scale of large language models (LLMs) necessitates highly efficient collective communication frameworks, particularly as training workloads extend to hundreds of tho…
Investigating Pedagogical Teacher and Student LLM Agents: Genetic Adaptation Meets Retrieval Augmented Generation Across Learning Style
Debdeep Sanyal, Agniva Maiti, Umakanta Maharana +4
Effective teaching requires adapting instructional strategies to accommodate the diverse cognitive and behavioral profiles of students, a persistent challenge in education and teac…