3 papers
cs.LG2026
OptRot: Mitigating Weight Outliers via Data-Free Rotations for Post-Training Quantization
Advait Gadhikar, Riccardo Grazzi, James Hensman
The presence of outliers in Large Language Models (LLMs) weights and activations makes them difficult to quantize. Recent work has leveraged rotations to mitigate these outliers. I…
cs.LG2024
TurboAttention: Efficient Attention Approximation For High Throughputs LLMs
Hao Kang, Srikant Bharadwaj, James Hensman +3
Large language model (LLM) inference demands significant amount of computation and memory, especially in the key attention mechanism. While techniques, such as quantization and acc…
stat.ML2024
Low-Rank Correction for Quantized LLMs
Meyer Scetbon, James Hensman
We consider the problem of model compression for Large Language Models (LLMs) at post-training time, where the task is to compress a well-trained model using only a small set of ca…