Fast-TurboQuant: A Multiplier-Free Online Vector Quantization Approach

Pedro M. R. Pereira, Felipe A. P. de Figueiredo, Rausley A. A. de Souza

As large language models scale, memory bandwidth for key-value caches and retrieval-augmented generation systems becomes a critical bottleneck. While 1-bit quantization addresses t…