31 citations · 51 across the 11 of their papers we have counts for
13 papers
KVStream: Adaptive Parallel Storage Management for KV Caching in Large-Scale LLM Serving
Avinash Maurya, Jaiaid Mobin, Jie Ye +3
Long-context LLM serving for agentic, coding, and document workloads reuses prefixes whose key-value (KV) cache outgrows GPU and host memory, putting storage I/O on the critical pa…
Towards Transparent Checkpointing with AI-driven Code Generation
Hai Duc Nguyen, Tekin Bicer, Kyle Chard +2
Adding reliable checkpoint/restart support to an MPI scientific application is a time-consuming expert effort that requires deep knowledge of both the application and resilience. W…
PTStore (Prefix Tensor Store): Distributed Prefix Caching and Replication for High Throughput Inference Serving
Meghana Maghyastha, Robert Underwood, Randal Burns +1
Inspired by the design of client caching in Content Delivery Networks (CDNs), PTStore distributes and replicates popular tensors that form reusable KV cache prefixes, which are the…
Recency/Frequency Adaptive KV Caching for Large Language Model Serving
Yang Shen, Meghana Madhyastha, Robert Underwood +2
Key-value (KV) caching is a powerful technique for accelerating large language model inference and generation. Inference workloads are large and diverse, which makes them difficult…
Beyond Fixed Budgets: Characterizing the Inelasticity and Limitations of Tree-of-Thought Reasoning Strategies
Atkia Mahila, Avinash Maurya, M. Mustafa Rafique +1
Tree of Thought (ToT) search has become a promising direction for improving the reasoning capabilities of large language models, but deploying these methods in practice raises a qu…
Understanding Inference Scaling for LLMs: Bottlenecks, Trade-offs, and Performance Principles
Moiz Arif, Avinash Maurya, Sudharshan Vazhkudai +1
The transition from standard generative AI to \emph{reasoning-centric architectures}, exemplified by models capable of extensive Chain-of-Thought~(CoT) processing, marks a fundamen…
DataStates-LLM: Scalable Checkpointing for Transformer Models Using Composable State Providers
Avinash Maurya, M. Mustafa Rafique, Franck Cappello +1
The rapid growth of Large Transformer-based models, specifically Large Language Models (LLMs), now scaling to trillions of parameters, has necessitated training across thousands of…
MLP-Offload: Multi-Level, Multi-Path Offloading for LLM Pre-training to Break the GPU Memory Wall
Avinash Maurya, M. Mustafa Rafique, Franck Cappello +1
Training LLMs larger than the aggregated memory of multiple GPUs is increasingly necessary due to the faster growth of LLM sizes compared to GPU memory. To this end, multi-tier hos…
EAIRA: Establishing a Methodology for Evaluating AI Models as Scientific Research Assistants
Franck Cappello, Sandeep Madireddy, Robert Underwood +23
Recent advancements have positioned AI, and particularly Large Language Models (LLMs), as transformative tools for scientific research, capable of addressing complex tasks that req…
Deep Optimizer States: Towards Scalable Training of Transformer Models Using Interleaved Offloading
Avinash Maurya, Jie Ye, M. Mustafa Rafique +2
Transformers and large language models~(LLMs) have seen rapid adoption in all domains. Their sizes have exploded to hundreds of billions of parameters and keep increasing. Under th…
Breaking the Memory Wall: A Study of I/O Patterns and GPU Memory Utilization for Hybrid CPU-GPU Offloaded Optimizers
Avinash Maurya, Jie Ye, M. Mustafa Rafique +2
Transformers and LLMs have seen rapid adoption in all domains. Their sizes have exploded to hundreds of billions of parameters and keep increasing. Under these circumstances, the t…
DataStates-LLM: Lazy Asynchronous Checkpointing for Large Language Models
Avinash Maurya, Robert Underwood, M. Mustafa Rafique +2
LLMs have seen rapid adoption in all domains. They need to be trained on high-end high-performance computing (HPC) infrastructures and ingest massive amounts of input data. Unsurpr…
Understanding Patterns of Deep Learning ModelEvolution in Network Architecture Search
Robert Underwood, Meghana Madhastha, Randal Burns +1
Network Architecture Search and specifically Regularized Evolution is a common way to refine the structure of a deep learning model.However, little is known about how models empiri…