Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Disaggregated Quantization: Specializing LLM Prefill and Decode
Andrei Panferov, Maximilian Kleinegger, Sweta Priyadarshi +2
Prefill and decode reward different approaches to quantization: low-precision arithmetic accelerates prompt processing, while compact weights reduce memory traffic during generatio…
cs.LG2026
MatGPTQ: Efficient and Accurate Inference over Nested Quantized Models
Maximilian Kleinegger, Elvir Crnčević, Dan Alistarh
Matryoshka Quantization (MatQuant), Any-Precision-LLM (AP) and AnyBCQ (AB) are recent quantization approaches showing that a single integer-quantized model can be served across mul…