NestQuant: Nested Lattice Quantization for Matrix Products and LLMs

Semyon Savkin, Eitan Porat, Or Ordentlich +1

Post-training quantization (PTQ) has emerged as a critical technique for efficient deployment of large language models (LLMs). This work proposes NestQuant, a novel PTQ scheme for…