Quantized Evolution Strategies: High-precision Fine-tuning of Quantized LLMs at Low-precision Cost

Yinggan Xu, Kajetan Schweighofer, Risto Miikkulainen +1

Post-Training Quantization (PTQ) is essential for deploying Large Language Models (LLMs) on memory-constrained devices, yet it renders models static and difficult to fine-tune. Sta…