22 citations · 22 across the 6 of their papers we have counts for
7 papers
Building py-kvcache: A Performance Characterization of External KV Caching for vLLM with NVMe SSDs
Joseph Kanichai, Tiziano De Matteis, Animesh Trivedi
Prefix caching can reduce the time to first token (TTFT) of long-context LLM requests by reusing previously computed key-value (KV) states, but for short prefixes or fast GPUs, rec…
DT-RAID: A Software-Defined Tiered RAID Architecture for Heterogeneous SSDs
Kun-Chi Chiang, Radu Stoica, Animesh Trivedi +5
The rapid proliferation of cloud and AI-driven workloads has led to increasingly complex requirements for modern storage subsystems. To meet these demands, SSD controller architect…
Breaking the Ice: Analyzing Cold Start Latency in vLLM
Huzaifa Shaaban Kabakibo, Animesh Trivedi, Lin Wang
As scalable inference services become popular, the cold start latency of an inference engine becomes important. Today, vLLM has evolved into the de-facto inference engine of choice…
Kavier: Exploring Performance, Sustainability, and Efficiency of LLM Ecosystems under Inference through Cache-Aware Discrete-Event Simulation
Radu Nicolae, Alexandru Iosup, Animesh Trivedi +1
Large Language Models (LLMs) are widely used by our increasingly digitalized society, but raise sustainability, performance, and financial concerns, especially as inference workloa…
Exploring GPU-to-GPU Communication: Insights into Supercomputer Interconnects
Daniele De Sensi, Lorenzo Pichetti, Flavio Vella +11
Multi-GPU nodes are increasingly common in the rapidly evolving landscape of exascale supercomputers. On these systems, GPUs on the same node are connected through dedicated networ…
Performance Characterization of NVMe Flash Devices with Zoned Namespaces (ZNS)
Krijn Doekemeijer, Nick Tehrany, Balakrishnan Chandrasekaran +2
The recent emergence of NVMe flash devices with Zoned Namespace support, ZNS SSDs, represents a significant new advancement in flash storage. ZNS SSDs introduce a new storage abstr…
Persistent Memory File Systems: A Survey
Wiebe van Breukelen, Animesh Trivedi
Persistent Memory (PM) is non-volatile byte-addressable memory that offers read and write latencies in the order of magnitude smaller than flash storage, such as SSDs. This survey…