28 papers
KernelSight-LM: A Kernel-Level LLM Inference Simulator
Xiteng Yao, Taeho Kim, Hengzhi Pei +7
As large language models (LLMs) move into production serving, practitioners must rapidly evaluate inference performance across diverse hardware, models, and serving parameters to m…
Optimizing CUDA like a Human: Micro-Profiling Tools as Expert Surrogates for LLM-Based GPU Kernel Optimization
Jiading Gai, Shuai Zhang, Kaj Bostrom +6
We present KernelPro, a closed-loop multi-agent system that automatically generates, profiles, and iteratively optimizes GPU kernel code by integrating large language model (LLM) c…
LLMZero: Discovering Adaptive Training Strategies for RL Post-Training via LLM Agents
Haoyang Fang, Wei Zhu, Boran Han +11
RL post-training strategies are dataset-dependent and reveal a recurring empirical pattern: capacity parameters accumulate monotonically across stages, while regularization paramet…
DualKV: Shared-Prompt Flash Attention for Efficient RL Training with Large Rollouts and Long Contexts
Jiading Gai, Shuai Zhang, Xiang Song +2
Modern RL post-training methods such as GRPO and DAPO train on N response sequences of R tokens sampled from a shared prompt of P tokens, but standard FlashAttention replicates all…
When LLMs get significantly worse: A statistical approach to detect model degradations
Jonas Kübler, Kailash Budhathoki, Matthäus Kleindessner +4
Minimizing the inference cost and latency of foundation models has become a crucial area of research. Optimization approaches include theoretically lossless methods and others with…
OPERA: Online Data Pruning for Efficient Retrieval Model Adaptation
Haoyang Fang, Shuai Zhang, Yifei Ma +5
Domain-specific finetuning is essential for dense retrievers, yet not all data pairs contribute equally to the learning process. We introduce OPERA, a data pruning framework that e…