6 papers · 1 filter
Large Language Models' Reasoning Stalls: An Investigation into the Capabilities of Frontier Models
Lachlan McGinness, Peter Baumgartner
Empirical methods to examine the capability of Large Language Models (LLMs) to use Automated Theorem Prover (ATP) reasoning strategies are studied. We evaluate the performance of S…
Large Language Models Imitate Logical Reasoning, but at what Cost?
Lachlan McGinness, Peter Baumgartner
We present a longitudinal study which evaluates the reasoning capability of frontier Large Language Models over an eighteen month period. We measured the accuracy of three leading…
The AlphaPhysics Term Rewriting System for Marking Algebraic Expressions in Physics Exams
Peter Baumgartner, Lachlan McGinness
We present our method for automatically marking Physics exams. The marking problem consists in assessing typed student answers for correctness with respect to a ground truth soluti…
Overview of AI Grading of Physics Olympiad Exams
Lachlan McGinness
Automatically grading the diverse range of question types in high school physics problem is a challenge that requires automated grading techniques from different fields. We report…
CON-FOLD -- Explainable Machine Learning with Confidence
Lachlan McGinness, Peter Baumgartner
FOLD-RM is an explainable machine learning classification algorithm that uses training data to create a set of classification rules. In this paper we introduce CON-FOLD which exten…
Automated Theorem Provers Help Improve Large Language Model Reasoning
Lachlan McGinness, Peter Baumgartner
In this paper we demonstrate how logic programming systems and Automated first-order logic Theorem Provers (ATPs) can improve the accuracy of Large Language Models (LLMs) for logic…