most citedFlattenQuant: Breaking Through the Inference Compute-bound for Large Language Models with Per-tensor Quantization

1 citations · 1 across the 1 of their papers we have counts for

collaborators

1 paper

cs.LG2024★ 1 cited

FlattenQuant: Breaking Through the Inference Compute-bound for Large Language Models with Per-tensor Quantization

Yi Zhang, Fei Yang, Shuang Peng +2

Large language models (LLMs) have demonstrated state-of-the-art performance across various tasks. However, the latency of inference and the large GPU memory consumption of LLMs res…