2 papers
cs.LG2025
Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deployment
Deokjae Lee, Hyun Oh Song
We study weight-only post-training quantization (PTQ), which quantizes the weights of a large language model (LLM) without retraining, using little or no calibration data. Weight-o…
cs.LG2025
GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance
Jinuk Kim, Marwa El Halabi, Wonpyo Park +5
Post-training quantization is a key technique for reducing the memory and inference latency of large language models by quantizing weights and activations without requiring retrain…