MXCode: Vector Quantization for MX-Native Inference at Scale

Gaurav Kolhe, Cansu Demirkiran, Shubhi Tyagi +8

Memory capacity and bandwidth are major bottlenecks in LLM inference, motivating weight compression beyond conventional 4-bit quantization. Vector quantization (VQ) enables approxi…