TSS: Target-Side Sparsification for Speculative Decoding in Domain-Specific Large Language Models

Haibo Hu, Lianming Huang, Qiao Li +2

Speculative decoding accelerates large language model inference through collaboration between a lightweight draft model and a target verifier. Existing methods mainly improve the d…