6 citations · 6 across the 6 of their papers we have counts for
5 papers · 1 filter
BOOST: Concurrent Access to Host Memory and HBM to Accelerate LLM Inference
Anish Saxena, Jae Hyung Ju, Hritvik Taneja +4
GPU memory bandwidth and capacity limit throughput in large language model (LLM) inference. The GPU memory system consists of a primary tier of high-bandwidth memory (HBM) and a se…
From Fleet to Lab: Revisiting the Security and Complexity of Industrial Rowhammer Mitigation
Hritvik Taneja, Moinuddin Qureshi
This paper studies efficient and secure Rowhammer mitigation at the Memory-Controller (MC). Rowhammer mitigation faces a fundamental tradeoff between tracking storage and mitigatio…
SiFAR: Synchronization-Free All-Reduce for Low-Latency LLM Inference
Hritvik Taneja, Anish Saxena, Abhishek Revinipati +3
The rise of reasoning models and agentic systems has made LLM token-generation latency a key bottleneck. Unlike chatbots, whose latency gains saturate at human reading speed, these…
TileLens: Efficiently Using Large-Granularity Memory Systems with Transparent Two-Dimensional Memory Layout
Jae Hyung Ju, Euijun Chung, Hritvik Taneja +4
Large Language Model (LLM) inference is bottlenecked by the capacity and bandwidth of GPU High-Bandwidth Memory (HBM). Recent proposals, such as High-Bandwidth Flash (HBF) and RoMe…
SIFT: Selective-Index For Fast Compute of RAG Prefill by Exploiting Attention Invariance
Rya Sanovar, Srikant Bharadwaj, Hritvik Taneja +1
Retrieval-Augmented Generation (RAG) injects LLM queries with relevant documents to improve response quality. This injection increases prompt length and slows time to first token (…