2 papers
cs.CL2024
A Survey on Efficient Inference for Large Language Models
Zixuan Zhou, Xuefei Ning, Ke Hong +12
Large Language Models (LLMs) have attracted extensive attention due to their remarkable performance across various tasks. However, the substantial computational and memory requirem…
cs.CL2024
Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation
Xuefei Ning, Zinan Lin, Zixuan Zhou +3
This work aims at decreasing the end-to-end generation latency of large language models (LLMs). One of the major causes of the high generation latency is the sequential decoding ap…