6.5k citations · 13.7k across the 24 of their papers we have counts for
24 papers
Regulating Branch Parallelism in LLM Serving
Swapnil Gandhi, Siva Hari, William J. Dally +1
Recent methods expose intra-request parallelism in LLM outputs, allowing independent branches to decode concurrently. Existing serving systems execute these branches eagerly or und…
Retrospective: EIE: Efficient Inference Engine on Sparse and Compressed Neural Network
Song Han, Xingyu Liu, Huizi Mao +4
EIE proposed to accelerate pruned and compressed neural networks, exploiting weight sparsity, activation sparsity, and 4-bit weight-sharing in neural network accelerators. Since pu…
SatIn: Hardware for Boolean Satisfiability Inference
Chenzhuo Zhu, Alexander C. Rucker, Yawen Wang +1
This paper describes SatIn, a hardware accelerator for determining boolean satisfiability (SAT) -- an important problem in many domains including verification, security analysis, a…
Optimal Clipping and Magnitude-aware Differentiation for Improved Quantization-aware Training
Charbel Sakr, Steve Dai, Rangharajan Venkatesan +3
Data clipping is crucial in reducing noise in quantization operations and improving the achievable accuracy of quantization-aware training (QAT). Current practices rely on heuristi…
GPU-Initiated On-Demand High-Throughput Storage Access in the BaM System Architecture
Zaid Qureshi, Vikram Sharma Mailthody, Isaac Gelado +10
Graphics Processing Units (GPUs) have traditionally relied on the host CPU to initiate access to the data storage. This approach is well-suited for GPU applications with known data…
PatchNet -- Short-range Template Matching for Efficient Video Processing
Huizi Mao, Sibo Zhu, Song Han +1
Object recognition is a fundamental problem in many video processing tasks, accurately locating seen objects at low computation cost paves the way for on-device video recognition.…