5 papers
Herschel: Continuous Optimization of Production LLM Inference through On-Demand Profiling
Luping Wang, Weigao Chen, Yifei Wu +11
Model-as-a-service platforms call for continuous optimization as complex serving conditions expose inefficiencies missed before deployment. Detailed always-on profiling can incur s…
Are LLM-Generated GPU Kernels Production-Ready? A Trace-Driven Benchmark and Optimization Agent
Lingyun Yang, Yuxiao Wang, Shenghao Liang +8
Existing GPU kernel generation benchmarks draw problems from synthetic or curated sources that diverge from deployed workloads. We present Atrex-Bench, a benchmark whose 30 operato…
WaveTune: Wave-aware Bilinear Modeling for Efficient GPU Kernel Auto-tuning
Kaixuan Zhang, Chutong Ding, Shiyou Qian +6
The rapid adoption of Large Language Models (LLMs) has made GPU inference efficiency an increasingly critical system concern. The runtime of LLM workloads is largely dominated by t…
PipeWeave: Synergizing Analytical and Learning Models for Unified GPU Performance Prediction
Kaixuan Zhang, Yunfan Cui, Shuhao Zhang +8
The rapid expansion of Transformer-based large language models has dramatically increased the need for high-performance GPUs. As a result, there is growing demand for fast, accurat…
EDGC: Entropy-driven Dynamic Gradient Compression for Efficient LLM Training
Qingao Yi, Jiaang Duan, Jun Zhang +11
Training large language models (LLMs) at scale incurs substantial communication overhead, while static gradient compression cannot adapt to gradient evolution and may degrade model…