Trainable Vector Quantization for Large Language Models

Anastasiia Prutianova, Alina Bogdanova, Mikhail Shutov +3

Vector Quantization (VQ) is a rapidly evolving technique for compressing Large Language Model (LLM) weights, reducing memory usage, and speeding up inference. However, extreme comp…