Rethinking Inference-Time Scaling: Efficiency Limits and Linguistic Signals

Junlin Wang, Shang Zhu, Jon Saad-Falcon +7

There is intense interest in investigating how inference time compute (ITC) (e.g. repeated sampling, refinements, etc) can improve large language model (LLM) capabilities. While br…