Fast Large Language Model Collaborative Decoding via Speculation

Jiale Fu, Yuchu Jiang, Junkai Chen +3

Large Language Model (LLM) collaborative decoding techniques improve output quality by combining the outputs of multiple models at each generation step, but they incur high computa…