Quantization of Large Language Models with an Overdetermined Basis

Daniil Merkulov, Daria Cherniuk, Alexander Rudikov +4

In this paper, we introduce an algorithm for data quantization based on the principles of Kashin representation. This approach hinges on decomposing any given vector, matrix, or te…