8 papers
SG-OPD: Sign-Gated On-Policy Distillation via Sign-Consistency Gating and Phased Teacher Sampling
Haoran Xu, Hongyu Wang, Yifei Gao +3
On-policy distillation (OPD) trains a student on its own trajectories with dense per-token supervision from a stronger teacher, and often outperforms off-policy distillation and st…
Visual Para-Thinker++: A Single-Policy Multi-Agent Framework for Visual Reasoning
Haoran Xu, Hongyu Wang, Yifei Gao +4
Visual reasoning requires integrating evidence distributed across regions, attributes, and relations, making single-chain reasoning prone to early perceptual commitment and halluci…
Reasoning Fails Where Step Flow Breaks
Xiaoyu Xu, Yulan Pan, Xiaosong Yuan +4
Large reasoning models (LRMs) that generate long chains of thought now perform well on multi-step math, science, and coding tasks. However, their behavior is still unstable and har…
ART: Attention Replacement Technique to Improve Factuality in LLMs
Ziqin Luo, Yihao Quan, Xiaofeng Zhang +2
Hallucination in large language models (LLMs) continues to be a significant issue, particularly in tasks like question answering, where models often generate plausible yet incorrec…
SalaMAnder: Shapley-based Mathematical Expression Attribution and Metric for Chain-of-Thought Reasoning
Yue Xin, Chen Shen, Shaotian Yan +5
Chain-of-Thought (CoT) prompting enhances the math reasoning capability of large language models (LLMs) to a large margin. However, the mechanism underlying such improvements remai…
Improving Complex Reasoning with Dynamic Prompt Corruption: A soft prompt Optimization Approach
Sinan Fan, Liang Xie, Chen Shen +7
Prompt-tuning (PT) for large language models (LLMs) can facilitate the performance on various conventional NLP tasks with significantly fewer trainable parameters. However, our inv…