Giving BERT a Calculator: Finding Operations and Arguments with Reading Comprehension

Abstract

Reading comprehension models have been successfully applied to extractive text answers, but it is unclear how best to generalize these models to abstractive numerical answers. We enable a BERT-based reading comprehension model to perform lightweight numerical reasoning. We augment the model with a predefined set of executable 'programs' which encompass simple arithmetic as well as extraction. Rather than having to learn to manipulate numbers directly, the model can pick a program and execute it. On the recent Discrete Reasoning Over Passages (DROP) dataset, designed to challenge reading comprehension models, we show a 33% absolute improvement by adding shallow programs. The model can learn to predict new operations when appropriate in a math word problem setting (Roy and Roth, 2015) with very few training examples.

References in corpus (1)

• ERNIE: Enhanced Representation through Knowledge Integration

Cited by in corpus (5)

• TAPAS: Weakly Supervised Table Parsing via Pre-training
• Neural Module Networks for Reasoning over Text
• Towards Question Format Independent Numerical Reasoning: A Set of Prerequisite Tasks
• A Reinforcement Learning Environment for Mathematical Reasoning via Program Synthesis
• An Empirical Investigation of Contextualized Number Prediction