VQ-Logits: Compressing the Output Bottleneck of Large Language Models via Vector Quantized Logits

Jintian Shao, Hongyi Huang, Jiayi Wu +4

Large Language Models (LLMs) have achieved remarkable success but face significant computational and memory challenges, particularly due to their extensive output vocabularies. The…