5 citations · 9 across the 22 of their papers we have counts for
Showing cs.DCShow all
3 papers · 1 filter
cs.DC2025
TinyServe: Query-Aware Cache Selection for Efficient LLM Serving
Dong Liu, Yanxuan Yu
Serving large language models (LLMs) efficiently remains challenging due to the high memory and latency overhead of key-value (KV) cache access during autoregressive decoding. We p…
cs.DC2025
PiKV: KV Cache Management System for Mixture of Experts
Dong Liu, Yanxuan Yu, Ben Lengerich +1
As large-scale language models continue to scale up in both size and context length, the memory and communication cost of key-value (KV) cache storage has become a major bottleneck…
cs.DC2024★ 5 cited
Designing Large Foundation Models for Efficient Training and Inference: A Survey
Dong Liu, Yanxuan Yu, Yite Wang +5
This paper focuses on modern efficient training and inference technologies on foundation models and illustrates them from two perspectives: model and system design. Model and Syste…