6 papers
QUADS: Stabilizing NVFP4 Reinforcement Learning for MoE via QUantization-error Alignment across Dual Sides
Zhengyang Zhuge, Hao Yu, Xin Wang +4
Rollout generation is a major bottleneck in Reinforcement Learning (RL) for Mixture-of-Experts (MoE) Large Language Models, motivating low-precision rollout acceleration such as FP…
Breaking Entropy Bounds: Accelerating RL Training via MTP with Rejection Sampling
Yucheng Li, Huiqiang Jiang, Yang Xu +14
Reinforcement learning (RL) has become a key component in modern large language models, yet the rollout stage remains the key bottleneck in RL training pipelines. Although Multi-To…
Overcoming Joint Intractability with Lossless Hierarchical Speculative Decoding
Yuxuan Zhou, Fei Huang, Heng Li +5
Verification is a key bottleneck in improving inference speed while maintaining distribution fidelity in Speculative Decoding. Recent work has shown that sequence-level verificatio…
TriSpec: Ternary Speculative Decoding via Lightweight Proxy Verification
Haoyun Jiang, Junqi He, Feng Hong +8
Inference efficiency in Large Language Models (LLMs) is fundamentally limited by their serial, autoregressive generation, especially as reasoning becomes a key capability and respo…
A Unified View of Attention and Residual Sinks: Outlier-Driven Rescaling is Essential for Transformer Training
Zihan Qiu, Zeyu Huang, Kaiyue Wen +16
We investigate the functional role of emergent outliers in large language models, specifically attention sinks (a few tokens that consistently receive large attention logits) and r…
VLCache: Computing 2% Vision Tokens and Reusing 98% for Vision-Language Inference
Shengling Qin, Hao Yu, Chenxin Wu +10
This paper presents VLCache, a cache reuse framework that exploits both Key-Value (KV) cache and encoder cache from prior multimodal inputs to eliminate costly recomputation when t…