8 papers
PLoRA: An NDP-Enhanced Pooled-Memory System for Cost-Efficient Multi-LoRA Serving
Zhongkai Yu, Ohm Rishabh Venkatachalam, Zheng Wang +9
Multi-LoRA serving is how one base model becomes thousands of specialized variants, one adapter per user, task, or agent, and the deployments can hold 1000-plus adapters. Serving t…
Syncopate: Efficient Multi-GPU AI Kernels via Automatic Chunk-Centric Compute-Communication Overlap
Xinwei Qiang, Yue Guan, Zhengding Hu +3
Communication has become a first-order bottleneck in large-scale GPU workloads, and existing distributed compilers address it mainly by overlapping whole compute and communication…
Patterns behind Chaos: Forecasting Data Movement for Efficient Large-Scale MoE LLM Inference
Zhongkai Yu, Yue Guan, Zihao Yu +6
Large-scale Mixture of Experts (MoE) Large Language Models (LLMs) have recently become the frontier open-weight models, achieving remarkable model capability similar to proprietary…
TritonDFT: Automating DFT with a Multi-Agent Framework
Zhengding Hu, Kuntal Talit, Zhen Wang +8
Density Functional Theory (DFT) is a cornerstone of materials science, yet executing DFT in practice requires coordinating a complex, multi-step workflow. Existing tools and LLM-ba…
Pancake: Hierarchical Memory System for Multi-Agent LLM Serving
Zhengding Hu, Zaifeng Pan, Prabhleen Kaur +6
In this work, we identify and address the core challenges of agentic memory management in LLM serving, where large-scale storage, frequent updates, and multiple coexisting agents j…
ChipBench: A Next-Step Benchmark for Evaluating LLM Performance in AI-Aided Chip Design
Zhongkai Yu, Chenyang Zhou, Yichen Lin +6
While Large Language Models (LLMs) show significant potential in hardware engineering, current benchmarks suffer from saturation and limited task diversity, failing to reflect LLMs…