AutoUVM: Automated Prefetching Framework for LLMs under UVM Oversubscription

Mao Lin, Hui Feng, Xianzhong Ding +3

Large language models (LLMs) increasingly exceed the memory capacity of commodity GPUs, making memory oversubscription common in practical deployments. NVIDIA Unified Virtual Memor…