6 papers
AReaL-TIK: Stateful Agentic Optimization of Unified RL Kernels through an Optimization IR
Ran Yan, Youhe Jiang, Jiayi Nie +5
Reinforcement learning (RL) post-training often uses distinct GPU kernels for rollout and policy update. In synchronous PPO and GRPO, numerical disagreement can perturb ratios betw…
When Does Disaggregation Pay? Simulating Prefill--Decode--Attention--FFN Specialization for Agentic LLM Inference
Przemyslaw Forys, Haoran Wu, Can Xiao +8
Agentic inference now dominates the LLM inference landscape, requiring LLMs to actively engage in multi-turn interactions with tool-calling capabilities. This introduces a more com…
MemExplorer: Navigating the Heterogeneous Memory Design Space for Agentic Inference NPUs
Haoran Wu, Zeyu Cao, Yao Lai +16
Emerging agentic large language model (LLM) workloads are driving rapidly growing demand for memory capacity and bandwidth. Different phases of inference, such as prefill and decod…
KernelCraft: Benchmarking for Agentic Close-to-Metal Kernel Generation on Emerging Hardware
Jiayi Nie, Haoran Wu, Yao Lai +9
New AI accelerators with novel instruction set architectures (ISAs) often require developers to manually craft low-level kernels, a time-consuming and error-prone process that does…
NPU Design for Diffusion Language Model Inference
Binglei Lou, Haoran Wu, Kevin Lau +9
Diffusion-based LLMs (dLLMs) fundamentally depart from traditional autoregressive (AR) LLM inference: they leverage bidirectional attention, block-wise KV cache refreshing, cross-s…
Combating the Memory Walls: Optimization Pathways for Long-Context Agentic LLM Inference
Haoran Wu, Can Xiao, Jiayi Nie +15
LLMs now form the backbone of AI agents across a diverse range of applications, including tool use, command-line interfaces, and web or computer interaction. These agentic LLM infe…