22 citations · 23 across the 3 of their papers we have counts for
3 papers
Building py-kvcache: A Performance Characterization of External KV Caching for vLLM with NVMe SSDs
Joseph Kanichai, Tiziano De Matteis, Animesh Trivedi
Prefix caching can reduce the time to first token (TTFT) of long-context LLM requests by reusing previously computed key-value (KV) states, but for short prefixes or fast GPUs, rec…
Exploring GPU-to-GPU Communication: Insights into Supercomputer Interconnects
Daniele De Sensi, Lorenzo Pichetti, Flavio Vella +11
Multi-GPU nodes are increasingly common in the rapidly evolving landscape of exascale supercomputers. On these systems, GPUs on the same node are connected through dedicated networ…
Streaming Task Graph Scheduling for Dataflow Architectures
Tiziano De Matteis, Lukas Gianinazzi, Johannes de Fine Licht +1
Dataflow devices represent an avenue towards saving the control and data movement overhead of Load-Store Architectures. Various dataflow accelerators have been proposed, but how to…