12 papers
PLoRA: An NDP-Enhanced Pooled-Memory System for Cost-Efficient Multi-LoRA Serving
Zhongkai Yu, Ohm Rishabh Venkatachalam, Zheng Wang +9
Multi-LoRA serving is how one base model becomes thousands of specialized variants, one adapter per user, task, or agent, and the deployments can hold 1000-plus adapters. Serving t…
FlashCP: Load-Balanced Communication-Efficient Context Parallelism for LLM Training
Zheng Wang, Eric Liu, Linan Jiang +5
Context parallelism (CP) is essential for training large-scale, long-context language models, as it partitions sequences to reduce memory overhead. However, existing CP methods suf…
SimSD: Simple Speculative Decoding in Diffusion Language Models
Junxia Cui, Haotian Ye, Runchu Tian +9
Diffusion large language models (dLLMs) have recently emerged as a promising alternative to autoregressive (AR) LLMs, offering faster inference through parallel or blockwise decodi…
ChipMATE: Multi-Agent Training via Reinforcement Learning for Enhanced RTL Generation
Zhongkai Yu, Yichen Lin, Chenyang Zhou +12
Existing API-based agentic systems for RTL code generation are fundamentally misaligned with industrial practice: they assume a golden testbench is available at generation time, re…
FlashEvolve: Accelerating Agent Self-Evolution with Asynchronous Stage Orchestration
Zhengding Hu, Mingge Lu, Zhen Wang +8
LLM-based evolution has emerged as a promising way to improve agents by refining non-parametric artifacts, but its wall-clock cost remains a major bottleneck. We identify that this…
AMMA: A Multi-Chiplet Memory-Centric Architecture for Low-Latency 1M Context Attention Serving
Zhongkai Yu, Haotian Ye, Chenyang Zhou +9
All current LLM serving systems place the GPU at the center, from production-level attention-FFN disaggregation to NVIDIA's Rubin GPU-LPU heterogeneous platform. Even academic PIM/…
JigsawRL: Assembling RL Pipelines for Efficient LLM Post-Training
Zhengding Hu, Hehua Ouyang, Chang Chen +6
We present JigsawRL, a cost-efficient framework that explores Pipeline Multiplexing as a new dimension of RL parallelism. JigsawRL decomposes each pipeline into a Sub-Stage Graph t…
Pancake: Hierarchical Memory System for Multi-Agent LLM Serving
Zhengding Hu, Zaifeng Pan, Prabhleen Kaur +6
In this work, we identify and address the core challenges of agentic memory management in LLM serving, where large-scale storage, frequent updates, and multiple coexisting agents j…
ChipBench: A Next-Step Benchmark for Evaluating LLM Performance in AI-Aided Chip Design
Zhongkai Yu, Chenyang Zhou, Yichen Lin +6
While Large Language Models (LLMs) show significant potential in hardware engineering, current benchmarks suffer from saturation and limited task diversity, failing to reflect LLMs…
ScaleSim: Serving Large-Scale Multi-Agent Simulation with Invocation Distance-Based Memory Management
Zaifeng Pan, Yipeng Shen, Zhengding Hu +6
LLM-based multi-agent simulations are increasingly adopted across application domains, but remain difficult to scale due to GPU memory pressure. Each agent maintains private GPU-re…
Patterns behind Chaos: Forecasting Data Movement for Efficient Large-Scale MoE LLM Inference
Zhongkai Yu, Yue Guan, Zihao Yu +6
Large-scale Mixture of Experts (MoE) Large Language Models (LLMs) have recently become the frontier open-weight models, achieving remarkable model capability similar to proprietary…
KPerfIR: Towards an Open and Compiler-centric Ecosystem for GPU Kernel Performance Tooling on Modern AI Workloads
Yue Guan, Yuanwei Fang, Keren Zhou +5
In this work, we propose KPerfIR, a novel multilevel compiler-centric infrastructure to enable the development of customizable, extendable, and portable profiling tools tailored fo…