3 papers
cs.LG2026
Disaggregated Quantization: Specializing LLM Prefill and Decode
Andrei Panferov, Maximilian Kleinegger, Sweta Priyadarshi +2
Prefill and decode reward different approaches to quantization: low-precision arithmetic accelerates prompt processing, while compact weights reduce memory traffic during generatio…
cs.PF2026
One Simple Trick for Improving the Performance of Energy-Limited Local Inference and Training
Erik Schultheis, Maximilian Kleinegger, Dan Alistarh
Energy supply and heat dissipation are two of the main challenges with modern GPU deployments. While typically discussed in the context of new datacenter constructions, the same co…
cs.LG2026
MatGPTQ: Efficient and Accurate Inference over Nested Quantized Models
Maximilian Kleinegger, Elvir Crnčević, Dan Alistarh
Matryoshka Quantization (MatQuant), Any-Precision-LLM (AP) and AnyBCQ (AB) are recent quantization approaches showing that a single integer-quantized model can be served across mul…