most citedAdaptive Configuration Selection for Multi-Model Inference Pipelines in Edge Computing

2 citations · 2 across the 2 of their papers we have counts for

collaborators
Showing cs.CLShow all

1 paper · 1 filter

cs.CL2026

SpecTr-GBV: Multi-Draft Block Verification Accelerating Speculative Decoding

Yijun Lin, Jinhao Sheng, Qingyue Cai +1

Autoregressive language models suffer from high inference latency due to their sequential decoding nature. Speculative decoding (SD) mitigates this by employing a lightweight draft…