2 papers
cs.DC2026
FlexNPU: Transparent NPU Virtualization for Dynamic LLM Prefill-Decode Co-location
Jiongjiong Gu, Jianfeng Wang, Zidong Han +19
Modern AI serving increasingly relies on NPUs for conventional inference and large language model serving. However, current NPU deployments commonly expose physical devices directl…
cs.DC2020
Simulating Performance of ML Systems with Offline Profiling
Hongming Huang, Peng Cheng, Hong Xu +1
We advocate that simulation based on offline profiling is a promising approach to better understand and improve the complex ML systems. Our approach uses operation-level profiling…