5 citations · 10 across the 18 of their papers we have counts for
6 papers · 1 filter
Prefix-Guided On-Policy Distillation: Mining Golden Trajectories from Rollouts
Qingfei Zhao, Huan Song, Shuyu Tian +2
On-policy distillation (OPD) improves reasoning models by applying dense teacher supervision on student-sampled trajectories. However, scaling OPD to long-horizon reasoning exposes…
Controllable Exploration in Hybrid-Policy RLVR for Multi-Modal Reasoning
Zhuoxu Huang, Mengxi Jia, Hao Sun +2
Reinforcement Learning with verifiable rewards (RLVR) has emerged as a primary learning paradigm for enhancing the reasoning capabilities of multi-modal large language models (MLLM…
Efficiently Aligning Draft Models via Parameter- and Data-Efficient Adaptation
Luxi Lin, Zhihang Lin, Zhanpeng Zeng +5
Speculative decoding accelerates LLM inference but suffers from performance degradation when target models are fine-tuned for specific domains. A naive solution is to retrain draft…
The Law of Multi-Model Collaboration: Scaling Limits of Model Ensembling for Large Language Models
Dakuan Lu, Jiaqi Zhang, Cheng Yuan +2
Recent advances in large language models (LLMs) have been largely driven by scaling laws for individual models, which predict performance improvements as model parameters and data…
Theoretical Foundations of Scaling Law in Familial Models
Huan Song, Qingfei Zhao, Ting Long +4
Neural scaling laws have become foundational for optimizing large language model (LLM) training, yet they typically assume a single dense model output. This limitation effectively…
CAS-Spec: Cascade Adaptive Self-Speculative Decoding for On-the-Fly Lossless Inference Acceleration of LLMs
Zhiyuan Ning, Jiawei Shao, Ruge Xu +4
Speculative decoding has become a widely adopted as an effective technique for lossless inference acceleration when deploying large language models (LLMs). While on-the-fly self-sp…