Transformers discover an elementary calculation system exploiting local attention and grid-like problem representation

Samuel Cognolato, Alberto Testolin

Mathematical reasoning is one of the most impressive achievements of human intellect but remains a formidable challenge for artificial intelligence systems. In this work we explore…