Solving Quantitative Reasoning Problems with Language Models

Abstract

Language models have achieved remarkable performance on a wide range of tasks that require natural language understanding. Nevertheless, state-of-the-art models have generally struggled with tasks that require quantitative reasoning, such as solving mathematics, science, and engineering problems at the college level. To help close this gap, we introduce Minerva, a large language model pretrained on general natural language data and further trained on technical content. The model achieves state-of-the-art performance on technical benchmarks without the use of external tools. We also evaluate our model on over two hundred undergraduate-level problems in physics, biology, chemistry, economics, and other sciences that require quantitative reasoning, and find that the model can correctly answer nearly a third of them.

arXiv comment12 pages, 5 figures + references and appendices

Cited by in corpus (11)

• Large language models in medicine: the potentials and pitfalls
• Unleashing the potential of prompt engineering for large language models
• Towards provably efficient quantum algorithms for large-scale machine-learning models
• A Survey on Symbolic Knowledge Distillation of Large Language Models
• Human I/O: Towards a Unified Approach to Detecting Situational Impairments
• Quantum Many-Body Physics Calculations with Large Language Models
• Symbolic Equation Solving via Reinforcement Learning
• Complex QA and language models hybrid architectures, Survey
• CoqPyt: Proof Navigation in Python in the Era of LLMs
• Learning to solve arithmetic problems with a virtual abacus
• Machine Reading Comprehension using Case-based Reasoning