QuIP: 2-Bit Quantization of Large Language Models With Guarantees

Jerry Chee, Yaohui Cai, Volodymyr Kuleshov +1

This work studies post-training parameter quantization in large language models (LLMs). We introduce quantization with incoherence processing (QuIP), a new method based on the insi…