WGRAMMAR: Leverage Prior Knowledge to Accelerate Structured Decoding

Ran Wang, Xiaoxuan Liu, Hao Ren +3

Structured decoding enables large language models (LLMs) to generate outputs in formats required by downstream systems, such as HTML or JSON. However, existing methods suffer from…