Adaptive Inference-Time Compute: LLMs Can Predict if They Can Do Better, Even Mid-Generation

Rohin Manvi, Anikait Singh, Stefano Ermon

Inference-time computation is a powerful paradigm to enhance the performance of large language models (LLMs), with Best-of-N sampling being a widely used technique. However, this m…