Dovetail: A CPU/GPU Heterogeneous Speculative Decoding for LLM inference

Libo Zhang, Zhaoning Zhang, Baizhou Xu +4

With the continuous advancement in the performance of large language models (LLMs), their demand for computational resources and memory has significantly increased, which poses maj…