6 papers · 1 filter
Beyond Teacher Assignment: Domain-Normalized Multi-Teacher On-Policy Distillation
Xin Li, Hao Jiang, Xin Gao +6
Reinforcement learning can turn one language model into several specialists, each excellent at a single skill such as mathematics, coding or following instructions, but users need…
ErrOPD: When Does On-Policy Distillation Transfer Teacher Errors?
Xin Li, Hao Jiang, Annan Wang +2
A distilled student can copy its teacher's mistakes along with its skills. On-policy distillation (OPD), used in recent post-training recipes, has the teacher grade every token of…
Towards Customized Multimodal Role-Play
Chao Tang, Jianzong Wu, Qingyu Shi +5
Unified multimodal understanding and generation models enable richer human-AI interaction. Yet jointly customizing a character's persona, dialogue style, and visual identity while…
Efficient Token-Level Policy Optimization for Masked Diffusion Models via Velocity Alignment
Chenxing Wei, Jiazhen Kang, Hong Wang +8
Reinforcement Learning with Verifiable Rewards (RLVR) successfully aligns autoregressive models in rigorous domains like mathematical reasoning and code generation. However, applyi…
Diffusion Language Models are Provably Optimal Parallel Samplers
Haozhe Jiang, Nika Haghtalab, Lijie Chen
Diffusion language models (DLMs) have emerged as a promising alternative to autoregressive models for faster inference via parallel token generation. We provide a rigorous foundati…
On Surjectivity of Neural Networks: Can you elicit any behavior from your model?
Haozhe Jiang, Nika Haghtalab
Given a trained neural network, can any specified output be generated by some input? Equivalently, does the network correspond to a function that is surjective? In generative model…