9 papers
FPTQuant: Function-Preserving Transforms for LLM Quantization
Boris van Breugel, Yelysei Bondarenko, Paul Whatmough +1
Large language models (LLMs) require substantial compute, and thus energy, at inference time. While quantizing weights and activations is effective at improving efficiency, naive q…
MobileWan: Closing the Quality Gap for Mobile Video Diffusion
Mohsen Ghafoorian, Denis Korzhenkov, Adil Karjauv +9
Recent advances in video diffusion have been driven by scaling transformer-based architectures to billions of parameters, substantially improving visual fidelity and motion coheren…
Leech Lattice Vector Quantization for Efficient LLM Compression
Tycho F. A. van der Ouderaa, Mart van Baalen, Paul Whatmough +1
Scalar quantization of large language models (LLMs) is fundamentally limited by information-theoretic bounds. While vector quantization (VQ) overcomes these limits by encoding bloc…
Dissecting Quantization Error: A Concentration-Alignment Perspective
Marco Federici, Boris van Breugel, Paul Whatmough +1
Quantization can drastically increase the efficiency of large language and vision models, but typically incurs an accuracy drop. Recently, function-preserving transforms (e.g. rota…
STaMP: Sequence Transformation and Mixed Precision for Low-Precision Activation Quantization
Marco Federici, Riccardo Del Chiaro, Boris van Breugel +2
Quantization is the key method for reducing inference latency, power and memory footprint of generative AI models. However, accuracy often degrades sharply when activations are qua…
HadaNorm: Diffusion Transformer Quantization through Mean-Centered Transformations
Marco Federici, Riccardo Del Chiaro, Boris van Breugel +2
Diffusion models represent the cutting edge in image generation, but their high memory and computational demands hinder deployment on resource-constrained devices. Post-Training Qu…