23 citations · 40 across the 10 of their papers we have counts for
Showing 2023Show all
2 papers · 1 filter
cs.LG2023
Fast and Efficient 2-bit LLM Inference on GPU: 2/4/16-bit in a Weight Matrix with Asynchronous Dequantization
Jinhao Li, Jiaming Xu, Shiyao Li +4
Large language models (LLMs) have demonstrated impressive abilities in various domains while the inference cost is expensive. Many previous studies exploit quantization methods to…
cs.LG2023★ 14 cited
FlashDecoding++: Faster Large Language Model Inference on GPUs
Ke Hong, Guohao Dai, Jiaming Xu +6
As the Large Language Model (LLM) becomes increasingly important in various domains. However, the following challenges still remain unsolved in accelerating LLM inference: (1) Sync…