FlexNPU: Transparent NPU Virtualization for Dynamic LLM Prefill-Decode Co-location

Jiongjiong Gu, Jianfeng Wang, Zidong Han +19

Modern AI serving increasingly relies on NPUs for conventional inference and large language model serving. However, current NPU deployments commonly expose physical devices directl…