Distinguishing Performance From Competence in Evaluations of Humanlike Abstract Reasoning

Claas Beger, Ryan Yi, Shuhao Fu +5

AI reasoning models have exceeded human performance on the ARC-AGI-1 benchmark, but does that mean state-of-the-art models have the underlying competence (humanlike abstract reason…