Latency and Token-Aware Test-Time Compute

Jenny Y. Huang, Mehul Damani, Yousef El-Kurdi +2

Inference-time scaling has emerged as a powerful way to improve large language model (LLM) performance by generating multiple candidate responses and selecting among them. However,…