2 citations · 2 across the 13 of their papers we have counts for
9 papers · 1 filter
Herschel: Continuous Optimization of Production LLM Inference through On-Demand Profiling
Luping Wang, Weigao Chen, Yifei Wu +11
Model-as-a-service platforms call for continuous optimization as complex serving conditions expose inefficiencies missed before deployment. Detailed always-on profiling can incur s…
Capture the lifecycle: KV Cache management in ReAct Agents with KVTether
Kaihua Fu, Yukun Zhou, Chaokun Chang +6
Efficient serving of long-context reasoning-and-acting (ReAct) agents relies on KV cache reuse to reduce large language model (LLM) prefill latency and monetary cost. However, a se…
Weave: Fine-Grained Dynamic SM Scheduling in an MoE Megakernel for Compute-Communication Overlap
Ziyu Huang, Yangjie Zhou, Chenhao Zhu +15
Mixture-of-Experts (MoE) inference under expert parallelism (EP) turns each MoE layer into a distributed computation with costly dispatch and combine communication. State-of-the-ar…
Are LLM-Generated GPU Kernels Production-Ready? A Trace-Driven Benchmark and Optimization Agent
Lingyun Yang, Yuxiao Wang, Shenghao Liang +8
Existing GPU kernel generation benchmarks draw problems from synthetic or curated sources that diverge from deployed workloads. We present Atrex-Bench, a benchmark whose 30 operato…
Ling and Ring 2.6 Technical Report: Efficient and Instant Agentic Intelligence at Trillion-Parameter Scale
Ang Li, Ben Liu, Bin Han +215
Efficient and scalable agentic intelligence requires models that can deliver both low-latency responses and strong reasoning capabilities while remaining practical to train, serve,…
WaveTune: Wave-aware Bilinear Modeling for Efficient GPU Kernel Auto-tuning
Kaixuan Zhang, Chutong Ding, Shiyou Qian +6
The rapid adoption of Large Language Models (LLMs) has made GPU inference efficiency an increasingly critical system concern. The runtime of LLM workloads is largely dominated by t…
LegoDiffusion: Micro-Serving Text-to-Image Diffusion Workflows
Lingyun Yang, Suyi Li, Tianyu Feng +10
Text-to-image generation executes a diffusion workflow comprising multiple models centered on a base diffusion model. Existing serving systems treat each workflow as an opaque mono…
Dissecting Outlier Dynamics in LLM NVFP4 Pretraining
Peijie Dong, Ruibo Fan, Yuechen Tao +11
Training large language models using 4-bit arithmetic enhances throughput and memory efficiency. Yet, the limited dynamic range of FP4 increases sensitivity to outliers. While NVFP…
PipeWeave: Synergizing Analytical and Learning Models for Unified GPU Performance Prediction
Kaixuan Zhang, Yunfan Cui, Shuhao Zhang +8
The rapid expansion of Transformer-based large language models has dramatically increased the need for high-performance GPUs. As a result, there is growing demand for fast, accurat…