Few Bits, One Law: Toward W2A4KV2

Kai Yi, Tarek Elgamal, Sruthikesh Surineni +3

Extreme low-bit LLM compression is most challenging when weights, activations, and KV caches are quantized together: their distributions differ, and quantization errors interact th…