Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
Contextures: Representations from Contexts
Runtian Zhai, Kai Yang, Che-Ping Tsai +3
Despite the empirical success of foundation models, we do not have a systematic characterization of the representations that these models learn. In this paper, we establish the con…
cs.LG2025
Spectral Journey: How Transformers Predict the Shortest Path
Andrew Cohen, Andrey Gromov, Kaiyu Yang +1
Decoder-only transformers lead to a step-change in capability of large language models. However, opinions are mixed as to whether they are really planning or reasoning. A path to m…