5 papers · 1 filter
Dual-Vocabulary Language Model for Cross-Tokenizer Distillation
Kedi Chen, Chen Lin, Yutao Sun +1
On-policy distillation (OPD) bridges teacher supervision and student behavior, but different teacher-student tokenizers introduce misalignment in both input tokenization (#1) and o…
ReNIO: Reweighting Negative Trajectory Importance for LLM On-Policy Distillation
Chen Lin, Kedi Chen, Wei Zhang
On-policy distillation (OPD) improves LLM reasoning by training a student model on its own generated outputs, but standard OPD treats all student-generated outputs (SGOs) equally r…
Rethinking Multiple-Choice Questions for RLVR: Unlocking Potential via Distractor Design
Xu Guo, Qiming Ge, Jian Tong +8
Reinforcement Learning with Verifiable Rewards (RLVR) significantly enhances the reasoning capabilities of Large Language Models. When applied to RLVR, Multiple-Choice Questions (M…
MERIT: Memory-Enhanced Retrieval for Interpretable Knowledge Tracing
Runze Li, Kedi Chen, Guwei Feng +3
Knowledge Tracing (KT) models students' evolving knowledge states to predict future performance, serving as a foundation for personalized education. While traditional deep learning…
Dynamic Multimodal Activation Steering for Hallucination Mitigation in Large Vision-Language Models
Jianghao Yin, Qin Chen, Kedi Chen +3
Large Vision-Language Models (LVLMs) exhibit outstanding performance on vision-language tasks but struggle with hallucination problems. Through in-depth analysis of LVLM activation…