Image-Space Rule Discovery

Abstract

Can image editors solve worksheets while preserving unrelated content? WISP (Worksheet Image-Space Problem Solving) defines 11 tasks and eight information conditions. On 2,200 shared items per editor, Nano Banana Pro achieves a 48.7% strict automatic proxy pass rate (Auto-Strict), versus 0.0-13.4% for four other editors. Analysis of 13,200 outputs, including a separate diagnostic, distinguishes target, preservation, and format failures. Auto-Strict agrees with human-majority judgments on 95.5% of 440 outputs (inter-rater Fleiss' $\kappa =0.894$). Excluding the two no-edit controls, Nano Banana Pro achieves 45.1% Auto-Strict, versus at most 3.2% for the other editors. Further probes show high symbol-tracking proxy scores but only 49% exact Wason selection. These results measure problem solving through edits, without establishing open-ended rule induction or transfer.

arXiv commentAccepted to ACCV 2026. Updated to the camera-ready version with minor clarifications, readability improvements, bibliographic updates, and a reformatted supplementary layout. 16 pages of main paper including references, plus 21 pages of supplementary material