8 papers
Why On-Policy Distillation Sometimes Fails: Vanishing Learning Signals
Lei Zhao, Qichao Zhao, Bowen Zuo +1
On-policy distillation (OPD) enables effective capability transfer between language models, yet the mechanisms underlying its failures are not fully understood. Across code generat…
Disentangling Language Roles in Multilingual LLM Task Execution
Qishi Zhan, Minxuan Hu, Seoyeon Jang +7
Multilingual LLMs are increasingly used when instruction, source content, and required response languages do not coincide. Existing benchmarks have expanded multilingual instructio…
TikArt: Stabilizing Aperture-Guided Fine-Grained Visual Reasoning with Reinforcement Learning
Hao Ding, Zhichuan Yang, Weijie Ge +3
Fine-grained visual reasoning in multimodal large language models (MLLMs) is bottlenecked by single-pass global image encoding: key evidence often lies in tiny objects, cluttered r…
LGM: Enhancing Large Language Models with Conceptual Meta-Relations and Iterative Retrieval
Wenchang Lei, Ping Zou, Yue Wang +2
Large language models (LLMs) exhibit strong semantic understanding, yet struggle when user instructions involve ambiguous or conceptually misaligned terms. We propose the Language…
A Model Can Help Itself: Reward-Free Self-Training for LLM Reasoning
Mengqi Li, Lei Zhao, Anthony Man-Cho So +2
Can a language model improve reasoning by learning from its own imperfect responses, without rewards or teacher-provided solutions? We present Self-evolving Post-Training (SePT), a…
StreamBP: Memory-Efficient Exact Backpropagation for Long Sequence Training of LLMs
Qijun Luo, Mengqi Li, Lei Zhao +1
Training language models on long sequence data is a demanding requirement for enhancing the model's capability on complex tasks, e.g., long-chain reasoning. However, as the sequenc…
DPO-Shift: Shifting the Distribution of Direct Preference Optimization
Xiliang Yang, Feng Jiang, Qianen Zhang +2
Direct Preference Optimization (DPO) and its variants have become increasingly popular for aligning language models with human preferences. These methods aim to teach models to bet…
How Transformers Utilize Multi-Head Attention in In-Context Learning? A Case Study on Sparse Linear Regression
Xingwu Chen, Lei Zhao, Difan Zou
Despite the remarkable success of transformer-based models in various real-world tasks, their underlying mechanisms remain poorly understood. Recent studies have suggested that tra…