Beam Search, Self-Consistency, and the Limits of Inference-Time Scaling for Grammar-Constrained Text-to-SQL in Small Language Models

Ty Chermsirivatana, John MacCormick

One common trade-off in the use of large language models involves reducing the size of the model while increasing the amount of computation at inference time, for example by using…