When the Canonical Completion Is Wrong: Formalizing and Measuring the Jump in Large Language Models

Abstract

Whether large language models (LLMs) can perform the abductive leap from evidence to a new system of axioms, commonly referred to as a jump, has recently attracted considerable debate. A prominent position holds that LLMs are structurally incapable of such jumps, while recent studies challenge both its mechanism and empirical evidence. One of the main reasons why the debate remains open is the lack of a formal definition of the jump and of algorithms that test whether a jump appears. In this paper, we attempt to develop a formal account of the jump in four steps and measure the second. These steps ask what the default completion of partial data is, when abandoning it is forced, whether the abandonment is correct, and how successive jumps compound. We define a \emph{jump instance} as a finite extension problem constrained by excluding the canonical completion given by the Kan extensions and leave exactly one unique correct completion. Under such an environment, a model offers a jump when it produces the correct completion, and fails to jump when it keeps the canonical default. We tested four frontier models on nine instances, where the Kan-default rate is zero in all 248 constrained trials, suggesting that the models can jump at least in this step. We further formalize the third and fourth steps, where a test of the jump is still wanted. We hope our work paves the path for formalizing and measuring the full jump in the future. The code of the paper is available at https://github.com/EEthanShi/kan-jump-test.