Mind the Gap: A Practical Attack on GGUF Quantization

Kazuki Egashira, Robin Staab, Mark Vero +2

With the increasing size of frontier LLMs, post-training quantization has become the standard for memory-efficient deployment. Recent work has shown that basic rounding-based quant…