Low-Rank Correction for Quantized LLMs

Meyer Scetbon, James Hensman

We consider the problem of model compression for Large Language Models (LLMs) at post-training time, where the task is to compress a well-trained model using only a small set of ca…