KBBQ: A Predictive Noise Law and the Limits of Spectrum Flattening in FP4 Quantization

Lexington Whalen, Yuki Ito, Ryo Sakamoto

We develop a second-order theory of quantization noise in matrix multiplication in which the quantization format is characterized by the variance it assigns to each element. The co…