6 papers
Bernoulli On-Policy Distillation: Don't Ignore High-Confidence Student Tokens
Mingyu Cao, Yaokai Jin, Shiwei Liu +1
On-policy distillation (OPD) trains a student language model on its own trajectories using token-level supervision from a stronger teacher. We show that standard sampled reverse-KL…
ValuSpec: Plug-and-Play Candidate Valuation before Target Verification for Tree-Based Speculative Decoding
Liang He, SiYuan Ma, Qishi Zhan +5
Tree-based speculative decoding accelerates large language model inference by verifying multiple candidate branches in parallel. However, in practical serving scenarios, similar re…
Beyond Correctness: Compute-Responsive Data Selection for Reasoning Model Post-Training
Liang He, Xiangyu Li, Yuxin Wang +4
Reasoning language models increasingly rely on test-time compute scaling, yet most existing work studies how to allocate inference computation while treating the model itself as fi…
Not All Visual Tokens Are Equally Safe to Remove:Consequence-Sensitive Visual Token Compression
Jingbo Wen, Liang He, Mingyu Cao +4
Visual token compression for vision--language models (VLMs) has largely relied on criteria such as attention, redundancy, and uncertainty to maximize average accuracy under a fixed…
Search or Accelerate: Confidence-Switched Position Beam Search for Diffusion Language Models
Mingyu Cao, Alvaro H. C. Correia, Christos Louizos +2
Diffusion Language Models (DLMs) generate text by iteratively denoising a masked sequence, repeatedly deciding which positions to commit at each step. Standard decoding follows a g…
Condense, Don't Just Prune: Enhancing Efficiency and Performance in MoE Layer Pruning
Mingyu Cao, Gen Li, Jie Ji +6
Mixture-of-Experts (MoE) has garnered significant attention for its ability to scale up neural networks while utilizing the same or even fewer active parameters. However, MoE does…