Spectral Journey: How Transformers Predict the Shortest Path

Andrew Cohen, Andrey Gromov, Kaiyu Yang +1

Decoder-only transformers lead to a step-change in capability of large language models. However, opinions are mixed as to whether they are really planning or reasoning. A path to m…