22 citations · 22 across the 6 of their papers we have counts for
4 papers · 1 filter
Building py-kvcache: A Performance Characterization of External KV Caching for vLLM with NVMe SSDs
Joseph Kanichai, Tiziano De Matteis, Animesh Trivedi
Prefix caching can reduce the time to first token (TTFT) of long-context LLM requests by reusing previously computed key-value (KV) states, but for short prefixes or fast GPUs, rec…
Kavier: Exploring Performance, Sustainability, and Efficiency of LLM Ecosystems under Inference through Cache-Aware Discrete-Event Simulation
Radu Nicolae, Alexandru Iosup, Animesh Trivedi +1
Large Language Models (LLMs) are widely used by our increasingly digitalized society, but raise sustainability, performance, and financial concerns, especially as inference workloa…
Exploring GPU-to-GPU Communication: Insights into Supercomputer Interconnects
Daniele De Sensi, Lorenzo Pichetti, Flavio Vella +11
Multi-GPU nodes are increasingly common in the rapidly evolving landscape of exascale supercomputers. On these systems, GPUs on the same node are connected through dedicated networ…
Performance Characterization of NVMe Flash Devices with Zoned Namespaces (ZNS)
Krijn Doekemeijer, Nick Tehrany, Balakrishnan Chandrasekaran +2
The recent emergence of NVMe flash devices with Zoned Namespace support, ZNS SSDs, represents a significant new advancement in flash storage. ZNS SSDs introduce a new storage abstr…