6 papers
Project Auto-World: Towards Automated Benchmarking of Neural Relational Reasoners
Anirban Das, Joanne Boisson, Irtaza Khalid +2
Reasoning about relational structures remains a significant challenge for neural models, particularly when they must systematically apply learned knowledge to problem instances tha…
Benchmarking Compositional Generalisation for Machine Learning Interatomic Potentials
Amir Masoud Nourollah, Irtaza Khalid, Stefano Leoni +1
Machine Learning Interatomic Potentials play a fundamental role in computational chemistry and materials science, enabling applications from molecular dynamics simulations to drug…
Shifting Perspectives: Steering Vectors for Robust Bias Mitigation in LLMs
Zara Siddique, Irtaza Khalid, Liam D. Turner +1
We present a novel approach to bias mitigation in large language models (LLMs) by applying steering vectors to modify model activations in forward passes. We compute 8 steering vec…
When No Paths Lead to Rome: Benchmarking Systematic Neural Relational Reasoning
Anirban Das, Irtaza Khalid, Rafael Peñaloza +1
Designing models that can learn to reason in a systematic way is an important and long-standing challenge. In recent years, a wide range of solutions have been proposed for the spe…
Large Language and Reasoning Models are Shallow Disjunctive Reasoners
Irtaza Khalid, Amir Masoud Nourollah, Steven Schockaert
Large Language Models (LLMs) have been found to struggle with systematic reasoning. Even on tasks where they appear to perform well, their performance often depends on shortcuts, r…
Systematic Relational Reasoning With Epistemic Graph Neural Networks
Irtaza Khalid, Steven Schockaert
Developing models that can learn to reason is a notoriously challenging problem. We focus on reasoning in relational domains, where the use of Graph Neural Networks (GNNs) seems li…