4 papers
When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning
Xiaogeng Liu, Xinyan Wang, Yingzi Ma +2
On-policy self-distillation (OPSD) trains a student on its own rollouts using a privileged teacher, but its standard objective weights all generated tokens equally, implicitly trea…
ASTRA-QA: A Benchmark for Abstract Question Answering over Documents
Shu Wang, Shansong Zhou, Xinyang Wang +3
Document-based question answering (QA) increasingly includes abstract questions that require synthesizing scattered information from long documents or across multiple documents int…
HaS: Accelerating RAG through Homology-Aware Speculative Retrieval
Peng Peng, Weiwei Lin, Wentai Wu +2
Retrieval-Augmented Generation (RAG) expands the knowledge boundary of large language models (LLMs) at inference by retrieving external documents as context. However, retrieval bec…
ROM: Real-time Overthinking Mitigation via Streaming Detection and Intervention
Xinyan Wang, Xiaogeng Liu, Ming Pei +1
Large Reasoning Models (LRMs) often reach a correct solution before their long Chain-of-Thought trace ends, yet continue with redundant verification, repeated attempts, or unnecess…