Internal Planning in Language Models: Characterizing Horizon and Branch Awareness

Muhammed Ustaomeroglu, Baris Askin, Gauri Joshi +2

The extent to which decoder-only language models (LMs) engage in planning, that is, organizing intermediate computations to support coherent long-range generation, remains an impor…