1 citations · 2 across the 7 of their papers we have counts for
3 papers · 1 filter
DEP: A Decentralized Large Language Model Evaluation Protocol
Jianxiang Peng, Junhao Li, Hongxiang Wang +15
With the rapid development of Large Language Models (LLMs), a large number of benchmarks have been proposed. However, most benchmarks lack unified evaluation standard and require t…
Make Every Draft Count: Hidden State based Speculative Decoding
Yuetao Chen, Xuliang Wang, Xinzhou Zheng +3
Speculative decoding has emerged as a pivotal technique to accelerate LLM inference by employing a lightweight draft model to generate candidate tokens that are subsequently verifi…
PRISM: Parametrically Refactoring Inference for Speculative Sampling Draft Models
Xuliang Wang, Yuetao Chen, Maochan Zhen +5
Large Language Models (LLMs), constrained by their auto-regressive nature, suffer from slow decoding. Speculative decoding methods have emerged as a promising solution to accelerat…