Quantization Beyond Uniform Bit Allocation

K. S. Sreeramji, Sabyasachi Basu, Ravishankar Krishnaswamy +2

Quantization is a fundamental technique to handle the growing sizes of embeddings generated by modern models. Existing quantization schemes are largely embedding agnostic and alloc…