Structured Transforms for Low-Overhead Quantization of Language Models

Daria Cherniuk, Alexander Rudikov, Boris Kashin +1

We revisit Kashin-decomposition-based weight quantization for large language models and propose an improved algorithm with stronger convergence properties and structured, efficient…