1 paper · 1 filter
Georgy Noarov, Soham Mallick, Tao Wang +5
Top-k decoding is a widely used method for sampling from LLMs: at each token, only the largest k next-token-probabilities are kept, and the next token is sampled after re-norma…