Quasi-random Multi-Sample Inference for Large Language Models

Aditya Parashar, Aditya Vikram Singh, Avinash Amballa +2

Large language models (LLMs) are often equipped with multi-sample decoding strategies. An LLM implicitly defines an arithmetic code book, facilitating efficient and embarrassingly…