1 citations · 1 across the 11 of their papers we have counts for
14 papers
AgentReplay: Token-Wise Trace Replay Is Essential for Fair Serving System Performance Benchmarking
Zaifeng Pan, Michael Wang, Chris Wu +4
LLM-based agents execute multi-turn workflows with interleaved model inference and tool calls, making efficient serving increasingly important. However, evaluating serving optimiza…
RR-Evict: Fine-Grained Prefix Cache Eviction beyond LRU for Agentic LLM Serving
Zaifeng Pan, Chris Wu, Zhengding Hu +3
LLM-based agents execute long-horizon tasks through repeated model calls interleaved with tool execution and user interaction. As each call extends the history accumulated in previ…
Beyond Parallel Blindness: Information Floors and Model Gaps in Block Drafting
Xinwei Qiang, Xiang Fang, Chang Chen +3
Block drafters propose several tokens in one forward pass, before earlier target tokens are realised. Their rejection mixes two losses: missing within-block path information and im…
SmoothAgent: Efficient Long-Horizon LLM-Based Agent Serving with Lookahead Context Engineering
Zaifeng Pan, Qianxu Wang, Zhengding Hu +5
LLM-based agents execute multi-turn workflows with continuously growing contexts, where LLM calls are interleaved with tool invocations and environment feedback. To maintain model…
FlashCP: Load-Balanced Communication-Efficient Context Parallelism for LLM Training
Zheng Wang, Eric Liu, Linan Jiang +5
Context parallelism (CP) is essential for training large-scale, long-context language models, as it partitions sequences to reduce memory overhead. However, existing CP methods suf…
TLX: Hardware-Native, Evolvable MIMW GPU Compiler for Large-scale Production Environments
Yue Guan, Hongtao Yu, Peng Chen +10
Modern GPUs increasingly rely on specialized hardware units and asynchronous coordination mechanisms, so performance depends on orchestrating data movement, tensor-core computation…
ChipMATE: Multi-Agent Training via Reinforcement Learning for Enhanced RTL Generation
Zhongkai Yu, Yichen Lin, Chenyang Zhou +12
Existing API-based agentic systems for RTL code generation are fundamentally misaligned with industrial practice: they assume a golden testbench is available at generation time, re…
FlashEvolve: Accelerating Agent Self-Evolution with Asynchronous Stage Orchestration
Zhengding Hu, Mingge Lu, Zhen Wang +8
LLM-based evolution has emerged as a promising way to improve agents by refining non-parametric artifacts, but its wall-clock cost remains a major bottleneck. We identify that this…
AMMA: A Multi-Chiplet Memory-Centric Architecture for Low-Latency 1M Context Attention Serving
Zhongkai Yu, Haotian Ye, Chenyang Zhou +9
All current LLM serving systems place the GPU at the center, from production-level attention-FFN disaggregation to NVIDIA's Rubin GPU-LPU heterogeneous platform. Even academic PIM/…
JigsawRL: Assembling RL Pipelines for Efficient LLM Post-Training
Zhengding Hu, Hehua Ouyang, Chang Chen +6
We present JigsawRL, a cost-efficient framework that explores Pipeline Multiplexing as a new dimension of RL parallelism. JigsawRL decomposes each pipeline into a Sub-Stage Graph t…
Yggdrasil: Bridging Dynamic Speculation and Static Runtime for Latency-Optimal Tree-Based LLM Decoding
Yue Guan, Changming Yu, Shihan Fang +8
Speculative decoding improves LLM inference by generating and verifying multiple tokens in parallel, but existing systems suffer from suboptimal performance due to a mismatch betwe…
HedraRAG: Coordinating LLM Generation and Database Retrieval in Heterogeneous RAG Serving
Zhengding Hu, Vibha Murthy, Zaifeng Pan +4
This paper addresses emerging system-level challenges in heterogeneous retrieval-augmented generation (RAG) serving, where complex multi-stage workflows and diverse request pattern…
KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows
Zaifeng Pan, Ajjkumar Patel, Zhengding Hu +6
Large language model (LLM) based agentic workflows have become a popular paradigm for coordinating multiple specialized agents to solve complex tasks. To improve serving efficiency…
WLB-LLM: Workload-Balanced 4D Parallelism for Large Language Model Training
Zheng Wang, Anna Cai, Xinfeng Xie +9
In this work, we present WLB-LLM, a workLoad-balanced 4D parallelism for large language model training. We first thoroughly analyze the workload imbalance issue in LLM training and…