RDQ: Residual Distribution Quantization for Large Language Models

Prateek Singh

Post-training quantization (PTQ) of large language models degrades sharply below 4-bit precision. We identify the root cause as residual stream distributional drift: quantization n…