Reclaiming Residual Knowledge: A Novel Paradigm to Low-Bit Quantization

Róisín Luo, Alexandru Drimbarean, James McDermott +1

This paper explores a novel paradigm in low-bit (i.e. 4-bits or lower) quantization, differing from existing state-of-the-art methods, by framing optimal quantization as an archite…