Evaluating Quantized Large Language Models

Shiyao Li, Xuefei Ning, Luning Wang +6

Post-training quantization (PTQ) has emerged as a promising technique to reduce the cost of large language models (LLMs). Specifically, PTQ can effectively mitigate memory consumpt…