4 papers
TeacherGRPO: Closing the Capacity Gap in Reasoning Distillation via Teacher Alignment
Zhenyu Lei, Zihan Chen, Yaochen Zhu +5
Reasoning distillation from powerful teacher models to smaller students faces the Gap Curse: as teachers grow more sophisticated, their complex distributions increasingly diverge f…
Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction
Zhuofeng Li, Haoxiang Zhang, Cong Wei +16
Modern retrieval systems, whether lexical or semantic, expose a corpus through a fixed similarity interface that compresses access into a single top-k retrieval step before reasoni…
Don't Throw Away Your Pretrained Model
Shangbin Feng, Wenhao Yu, Yike Wang +3
Alignment training has tradeoffs: it helps language models (LMs) gain in reasoning and instruction following but might lose out on skills such as creativity and calibration, where…
Bidirectional LMs are Better Knowledge Memorizers? A Benchmark for Real-world Knowledge Injection
Yuwei Zhang, Wenhao Yu, Shangbin Feng +5
Despite significant advances in large language models (LLMs), their knowledge memorization capabilities remain underexplored, due to the lack of standardized and high-quality test…