3 papers
cs.AR2025
Accelerating LLM Inference via Dynamic KV Cache Placement in Heterogeneous Memory System
Yunhua Fang, Rui Xie, Asad Ul Haq +6
Large Language Model (LLM) inference is increasingly constrained by memory bandwidth, with frequent access to the key-value (KV) cache dominating data movement. While attention spa…
cs.DB2024
ZipCache: A DRAM/SSD Cache with Built-in Transparent Compression
Rui Xie, Linsen Ma, Alex Zhong +2
As a core component in modern data centers, key-value cache provides high-throughput and low-latency services for high-speed data processing. The effectiveness of a key-value cache…
cs.LG2024
SmartQuant: CXL-based AI Model Store in Support of Runtime Configurable Weight Quantization
Rui Xie, Asad Ul Haq, Linsen Ma +5
Recent studies have revealed that, during the inference on generative AI models such as transformer, the importance of different weights exhibits substantial context-dependent vari…