2 papers
cs.CL2026
BLASST: Dynamic BLocked Attention Sparsity via Softmax Thresholding
Jiayi Yuan, Cameron Shinn, Kai Xu +19
The growing demand for long-context inference capabilities in Large Language Models (LLMs) has intensified the computational and memory bottlenecks inherent to the self-attention m…
cs.CL2024
SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
Guangxuan Xiao, Ji Lin, Mickael Seznec +3
Large language models (LLMs) show excellent performance but are compute- and memory-intensive. Quantization can reduce memory and accelerate inference. However, existing methods ca…