2 citations · 2 across the 3 of their papers we have counts for
1 paper · 1 filter
Bangsheng Tang, Carl Chengyan Fu, Fei Kou +35
Speculative decoding is a standard method for accelerating the inference speed of large language models. However, scaling it for production environments poses several engineering c…