Universal Length Generalization with Turing Programs

Kaiying Hou, David Brandfonbrener, Sham Kakade +2

Length generalization refers to the ability to extrapolate from short training sequences to long test sequences and is a challenge for current large language models. While prior wo…