How predictable is language model benchmark performance?

David Owen

We investigate large language model performance across five orders of magnitude of compute scaling in eleven recent model architectures. We show that average benchmark performance,…