QuantX: A Framework for Hardware-Aware Quantization of Generative AI Workloads

Muhammad Ahmad, Khurram Mazher, Saqib Akram +2

We present QuantX: a tailored suite of recipes for LLM and VLM quantization. It is capable of quantizing down to 3-bit resolutions with minimal loss in performance. The quantizatio…