DPS: Dual-Mode Precision LLM Serving with Semi-Unified Memory

Xuan Truong Nguyen, Tien Son Pham, Tuan Duc Chu +2

Existing LLM serving systems virtualize and optimize KV-cache memory, but treat model-weight memory as fixed throughout execution. Recent work on multi-precision model representati…