PreSTO: Predictive Subtree Prefetching for Fast LLM Power Sampling

Nan Jiang, Panagiotis Theodoropoulos, Wei Deng

Power sampling improves large language model (LLM) reasoning at test time by sampling from a sharpened version of the base model's distribution. Its sampler, PowerMH, is slow becau…