5 papers
KernelSight-LM: A Kernel-Level LLM Inference Simulator
Xiteng Yao, Taeho Kim, Hengzhi Pei +7
As large language models (LLMs) move into production serving, practitioners must rapidly evaluate inference performance across diverse hardware, models, and serving parameters to m…
DualKV: Shared-Prompt Flash Attention for Efficient RL Training with Large Rollouts and Long Contexts
Jiading Gai, Shuai Zhang, Xiang Song +2
Modern RL post-training methods such as GRPO and DAPO train on N response sequences of R tokens sampled from a shared prompt of P tokens, but standard FlashAttention replicates all…
XShare: Collaborative in-Batch Expert Sharing for Faster MoE Inference
Daniil Vankov, Nikita Ivkin, Kyle Ulrich +3
Mixture-of-Experts (MoE) architectures are increasingly used to efficiently scale large language models. However, in production inference, request batching and speculative decoding…
P-EAGLE: Parallel-Drafting EAGLE with Scalable Training
Mude Hui, Xin Huang, Jaime Campos Salas +5
Reasoning LLMs produce longer outputs, requiring speculative decoding drafters trained on extended sequences. Parallel drafting - predicting multiple tokens per forward pass - offe…
Neural HD Map Generation from Multiple Vectorized Tiles Locally Produced by Autonomous Vehicles
Miao Fan, Yi Yao, Jianping Zhang +2
High-definition (HD) map is a fundamental component of autonomous driving systems, as it can provide precise environmental information about driving scenes. Recent work on vectoriz…