93 citations · 446 across the 32 of their papers we have counts for
Showing cs.CLShow all
2 papers · 1 filter
cs.CL2026
Procedural Pretraining: Warming Up Language Models with Abstract Data
Liangze Jiang, Zachary Shinnick, Anton van den Hengel +2
Pretraining language models directly on web-scale corpora is the de facto paradigm. We study an alternative where the model is initially exposed to abstract structured data to ease…
cs.CL2023
A Symbolic Framework for Evaluating Mathematical Reasoning and Generalisation with Transformers
Jordan Meadows, Marco Valentino, Damien Teney +1
This paper proposes a methodology for generating and perturbing detailed derivations of equations at scale, aided by a symbolic engine, to evaluate the generalisability of Transfor…