Watermarkable Multi-Draft Speculative Sampling via Poisson Processes

Yanxiao Liu, Sicheng Wan, Zhan Gao +1

Large language models (LLMs) have achieved state-of-the-art performance across a wide range of tasks, motivating two important aspects of deployment: inference efficiency and outpu…