Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation

Xuefei Ning, Zinan Lin, Zixuan Zhou +3

This work aims at decreasing the end-to-end generation latency of large language models (LLMs). One of the major causes of the high generation latency is the sequential decoding ap…