6 papers · 1 filter
On-Policy Residual Learning for Diffusion Language Models
Yifan Yu, Yuqing Jian, Jisen Li +9
Residual context is key to unlocking the speed-quality Pareto frontier of diffusion language models (DLMs) by carrying forward predictive distributions across denoising steps. Yet…
QATFactory: A Versatile, Deployment-Aligned Framework for Quantization-aware Training and Distillation of LLMs
Weili Xu, Jisen Li, Yuqing Jian +8
Large language model (LLM) inference is increasingly moving toward lower precision to realize the throughput of hardware accelerators, but aggressive post-training quantization (PT…
Tail-Likelihood Reinforcement Learning
Shrinivas Ramasubramanian, Daman Arora, Fahim Tajwar +11
Reinforcement learning typically optimizes average reward. For generative policies, the average can hide an important distinction: two policies can achieve the same mean reward whi…
Osprey: Target-agnostic Pre-training Makes Stronger Drafters in Speculative Decoding
Fengxiang Bie, Yuqing Jian, Yifan Yu +7
Speculative decoding is critical for accelerating LLM inference. However, the speedup is fragile: drafters are typically trained against a narrow distribution for a single target m…
ThunderAgent: A Simple, Fast and Program-Aware Agentic Inference System
Hao Kang, Ziyang Li, Weili Xu +7
Large language models(LLMs) are now used to power complex multi-turn agentic workflows. Existing systems run agentic inference by loosely assembling isolated components: an LLM inf…
When RL Meets Adaptive Speculative Training: A Unified Training-Serving System
Junxiong Wang, Fengxiang Bie, Jisen Li +14
Speculative decoding can significantly accelerate LLM serving, yet most deployments today disentangle speculator training from serving, treating speculator training as a standalone…