MatheMagic: Generating Dynamic Mathematics Benchmarks Robust to Memorization

Dayyán O'Brien, Barry Haddow, Emily Allaway +1

Conducting contamination-free evaluation of mathematical capabilities can be difficult for two reasons: models may memorize a test set once it is made public, and current mathemati…