FPTQuant: Function-Preserving Transforms for LLM Quantization

Boris van Breugel, Yelysei Bondarenko, Paul Whatmough +1

Large language models (LLMs) require substantial compute, and thus energy, at inference time. While quantizing weights and activations is effective at improving efficiency, naive q…