Arithmetic Without Algorithms: Language Models Solve Math With a Bag of Heuristics

Yaniv Nikankin, Anja Reusch, Aaron Mueller +1

Do large language models (LLMs) solve reasoning tasks by learning robust generalizable algorithms, or do they memorize training data? To investigate this question, we use arithmeti…