Can neural networks do arithmetic? A survey on the elementary numerical skills of state-of-the-art deep learning models

Abstract

Creating learning models that can exhibit sophisticated reasoning skills is one of the greatest challenges in deep learning research, and mathematics is rapidly becoming one of the target domains for assessing scientific progress in this direction. In the past few years there has been an explosion of neural network architectures, data sets, and benchmarks specifically designed to tackle mathematical problems, reporting notable success in disparate fields such as automated theorem proving, numerical integration, and discovery of new conjectures or matrix multiplication algorithms. However, despite these impressive achievements it is still unclear whether deep learning models possess an elementary understanding of quantities and symbolic numbers. In this survey we critically examine the recent literature, concluding that even state-of-the-art architectures often fall short when probed with relatively simple tasks designed to test basic numerical and arithmetic knowledge.

References in corpus (16)

• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
• The Symbol Grounding Problem
• Measuring Mathematical Problem Solving With the MATH Dataset
• A Neural Network Solves, Explains, and Generates University Math Problems by Program Synthesis and Few-Shot Learning at Human Level
• MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms
• DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
• Analysing Mathematical Reasoning Abilities of Neural Models
• Bias and Generalization in Deep Generative Models: An Empirical Study
• Exploring Length Generalization in Large Language Models
• Discrete and continuous representations and processing in deep learning: Looking forward
• Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problems
• Show Your Work: Scratchpads for Intermediate Computation with Language Models
• Learning to Prove Theorems by Learning to Generate Theorems
• The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization
• Learning to solve arithmetic problems with a virtual abacus
• A Hybrid System for Systematic Generalization in Simple Arithmetic Problems

Cited by in corpus (1)

• Calabi-Yau Four/Five/Six-folds as ${\mathbb{P}}_{\text{w}}^n$ Hypersurfaces: Machine Learning, Approximation, and Generation