1 citations · 1 across the 12 of their papers we have counts for
16 papers
AgentReplay: Token-Wise Trace Replay Is Essential for Fair Serving System Performance Benchmarking
Zaifeng Pan, Michael Wang, Chris Wu +4
LLM-based agents execute multi-turn workflows with interleaved model inference and tool calls, making efficient serving increasingly important. However, evaluating serving optimiza…
RR-Evict: Fine-Grained Prefix Cache Eviction beyond LRU for Agentic LLM Serving
Zaifeng Pan, Chris Wu, Zhengding Hu +3
LLM-based agents execute long-horizon tasks through repeated model calls interleaved with tool execution and user interaction. As each call extends the history accumulated in previ…
SmoothAgent: Efficient Long-Horizon LLM-Based Agent Serving with Lookahead Context Engineering
Zaifeng Pan, Qianxu Wang, Zhengding Hu +5
LLM-based agents execute multi-turn workflows with continuously growing contexts, where LLM calls are interleaved with tool invocations and environment feedback. To maintain model…
Demystifying Training-Time Augmentation for Data-Constrained Language Model Pretraining
Michael K. Chen, Xikun Zhang, Fan Bai +2
As AI labs approach a data ceiling where compute capacity outpaces the rate of new high-quality text generation, language model pretraining is shifting toward a data-constrained, c…
TLX: Hardware-Native, Evolvable MIMW GPU Compiler for Large-scale Production Environments
Yue Guan, Hongtao Yu, Peng Chen +10
Modern GPUs increasingly rely on specialized hardware units and asynchronous coordination mechanisms, so performance depends on orchestrating data movement, tensor-core computation…
ChipMATE: Multi-Agent Training via Reinforcement Learning for Enhanced RTL Generation
Zhongkai Yu, Yichen Lin, Chenyang Zhou +12
Existing API-based agentic systems for RTL code generation are fundamentally misaligned with industrial practice: they assume a golden testbench is available at generation time, re…
FlashEvolve: Accelerating Agent Self-Evolution with Asynchronous Stage Orchestration
Zhengding Hu, Mingge Lu, Zhen Wang +8
LLM-based evolution has emerged as a promising way to improve agents by refining non-parametric artifacts, but its wall-clock cost remains a major bottleneck. We identify that this…
AMMA: A Multi-Chiplet Memory-Centric Architecture for Low-Latency 1M Context Attention Serving
Zhongkai Yu, Haotian Ye, Chenyang Zhou +9
All current LLM serving systems place the GPU at the center, from production-level attention-FFN disaggregation to NVIDIA's Rubin GPU-LPU heterogeneous platform. Even academic PIM/…
JigsawRL: Assembling RL Pipelines for Efficient LLM Post-Training
Zhengding Hu, Hehua Ouyang, Chang Chen +6
We present JigsawRL, a cost-efficient framework that explores Pipeline Multiplexing as a new dimension of RL parallelism. JigsawRL decomposes each pipeline into a Sub-Stage Graph t…
TritonDFT: Automating DFT with a Multi-Agent Framework
Zhengding Hu, Kuntal Talit, Zhen Wang +8
Density Functional Theory (DFT) is a cornerstone of materials science, yet executing DFT in practice requires coordinating a complex, multi-step workflow. Existing tools and LLM-ba…
Pancake: Hierarchical Memory System for Multi-Agent LLM Serving
Zhengding Hu, Zaifeng Pan, Prabhleen Kaur +6
In this work, we identify and address the core challenges of agentic memory management in LLM serving, where large-scale storage, frequent updates, and multiple coexisting agents j…
ScaleSim: Serving Large-Scale Multi-Agent Simulation with Invocation Distance-Based Memory Management
Zaifeng Pan, Yipeng Shen, Zhengding Hu +6
LLM-based multi-agent simulations are increasingly adopted across application domains, but remain difficult to scale due to GPU memory pressure. Each agent maintains private GPU-re…
Syncopate: Efficient Multi-GPU AI Kernels via Automatic Chunk-Centric Compute-Communication Overlap
Xinwei Qiang, Yue Guan, Zhengding Hu +3
Communication has become a first-order bottleneck in large-scale GPU workloads, and existing distributed compilers address it mainly by overlapping whole compute and communication…
Patterns behind Chaos: Forecasting Data Movement for Efficient Large-Scale MoE LLM Inference
Zhongkai Yu, Yue Guan, Zihao Yu +6
Large-scale Mixture of Experts (MoE) Large Language Models (LLMs) have recently become the frontier open-weight models, achieving remarkable model capability similar to proprietary…
HedraRAG: Coordinating LLM Generation and Database Retrieval in Heterogeneous RAG Serving
Zhengding Hu, Vibha Murthy, Zaifeng Pan +4
This paper addresses emerging system-level challenges in heterogeneous retrieval-augmented generation (RAG) serving, where complex multi-stage workflows and diverse request pattern…
KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows
Zaifeng Pan, Ajjkumar Patel, Zhengding Hu +6
Large language model (LLM) based agentic workflows have become a popular paradigm for coordinating multiple specialized agents to solve complex tasks. To improve serving efficiency…