Fast LLM Post-training via Decoupled and Fastest-of-N Speculation

Rongxin Cheng, Kai Zhou, Xingda Wei +8

Rollout dominates the training time in large language model (LLM) post-training, where the trained model is used to generate tokens given a batch of prompts. This work, SpecActor,…