2 papers
cs.LG2025
ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantization
Zechun Liu, Changsheng Zhao, Hanxian Huang +13
The optimal bit-width for achieving the best trade-off between quantized model size and accuracy has been a subject of ongoing debate. While some advocate for 4-bit quantization, o…
cs.DC2024
HadaCore: Tensor Core Accelerated Hadamard Transform Kernel
Krish Agarwal, Rishi Astra, Adnan Hoque +4
We present HadaCore, a modified Fast Walsh-Hadamard Transform (FWHT) algorithm optimized for the Tensor Cores present in modern GPU hardware. HadaCore follows the recursive structu…