Showing cs.DCShow all
2 papers · 1 filter
cs.DC2026
BOOST: Concurrent Access to Host Memory and HBM to Accelerate LLM Inference
Anish Saxena, Jae Hyung Ju, Hritvik Taneja +4
GPU memory bandwidth and capacity limit throughput in large language model (LLM) inference. The GPU memory system consists of a primary tier of high-bandwidth memory (HBM) and a se…
cs.DC2026
SiFAR: Synchronization-Free All-Reduce for Low-Latency LLM Inference
Hritvik Taneja, Anish Saxena, Abhishek Revinipati +3
The rise of reasoning models and agentic systems has made LLM token-generation latency a key bottleneck. Unlike chatbots, whose latency gains saturate at human reading speed, these…