PhaseGate: Phase-Aware CPU Retrieval Scheduling for On-Device LLMs on Unified Memory

Seoyoon Yum, Sehoon Kim

On-device assistants run GPU-based LLM inference alongside CPU retrieval on unified-memory systems. Under a saturated local-retrieval workload, four concurrent retrieval workers ra…