9 papers
BOOST: Concurrent Access to Host Memory and HBM to Accelerate LLM Inference
Anish Saxena, Jae Hyung Ju, Hritvik Taneja +4
GPU memory bandwidth and capacity limit throughput in large language model (LLM) inference. The GPU memory system consists of a primary tier of high-bandwidth memory (HBM) and a se…
SiFAR: Synchronization-Free All-Reduce for Low-Latency LLM Inference
Hritvik Taneja, Anish Saxena, Abhishek Revinipati +3
The rise of reasoning models and agentic systems has made LLM token-generation latency a key bottleneck. Unlike chatbots, whose latency gains saturate at human reading speed, these…
TileLens: Efficiently Using Large-Granularity Memory Systems with Transparent Two-Dimensional Memory Layout
Jae Hyung Ju, Euijun Chung, Hritvik Taneja +4
Large Language Model (LLM) inference is bottlenecked by the capacity and bandwidth of GPU High-Bandwidth Memory (HBM). Recent proposals, such as High-Bandwidth Flash (HBF) and RoMe…
Utility-Driven Speculative Decoding for Mixture-of-Experts
Anish Saxena, Po-An Tsai, Hritvik Taneja +2
GPU memory bandwidth is the main bottleneck for low-latency Large Language Model (LLM) inference. Speculative decoding leverages idle GPU compute by using a lightweight drafter to…
Preventing Rowhammer Exploits via Low-Cost Domain-Aware Memory Allocation
Anish Saxena, Walter Wang, Alexandros Daglis
Rowhammer is a hardware security vulnerability at the heart of every system with modern DRAM-based memory. Despite its discovery a decade ago, comprehensive defenses remain elusive…
ImPress: Securing DRAM Against Data-Disturbance Errors via Implicit Row-Press Mitigation
Moinuddin Qureshi, Anish Saxena, Aamer Jaleel
DRAM cells are susceptible to Data-Disturbance Errors (DDE), which can be exploited by an attacker to compromise system security. Rowhammer is a well-known DDE vulnerability that o…
Randomized Line-to-Row Mapping for Low-Overhead Rowhammer Mitigations
Anish Saxena, Saurav Mathur, Moinuddin Qureshi
Modern systems mitigate Rowhammer using victim refresh, which refreshes the two neighbours of an aggressor row when it encounters a specified number of activations. Unfortunately,…
Scalable and Configurable Tracking for Any Rowhammer Threshold
Anish Saxena, Moinuddin Qureshi
The Rowhammer vulnerability continues to get worse, with the Rowhammer Threshold (TRH) reducing from 139K activations to 4.8K activations over the last decade. Typical Rowhammer mi…
A Case for CXL-Centric Server Processors
Albert Cho, Anish Saxena, Moinuddin Qureshi +1
The memory system is a major performance determinant for server processors. Ever-growing core counts and datasets demand higher bandwidth and capacity as well as lower latency from…