7 papers
Knowing the Facts but Choosing the Shortcut: Understanding How Large Language Models Compare Entities
Hans Hergen Lehmann, Jae Hee Lee, Steven Schockaert +1
Large Language Models (LLMs) are increasingly used for knowledge-based reasoning tasks, yet understanding when they rely on genuine knowledge versus superficial heuristics remains…
Extracting Conceptual Spaces from LLMs Using Prototype Embeddings
Nitesh Kumar, Usashi Chatterjee, Steven Schockaert
Conceptual spaces represent entities and concepts using cognitively meaningful dimensions, typically referring to perceptual features. Such representations are widely used in cogni…
Faithful Differentiable Reasoning with Reshuffled Region-based Embeddings
Aleksandar Pavlovic, Emanuel Sallinger, Steven Schockaert
Knowledge graph (KG) embedding methods learn geometric representations of entities and relations to predict plausible missing knowledge. These representations are typically assumed…
Large Language and Reasoning Models are Shallow Disjunctive Reasoners
Irtaza Khalid, Amir Masoud Nourollah, Steven Schockaert
Large Language Models (LLMs) have been found to struggle with systematic reasoning. Even on tasks where they appear to perform well, their performance often depends on shortcuts, r…
Ragas: Automated Evaluation of Retrieval Augmented Generation
Shahul Es, Jithin James, Luis Espinosa-Anke +1
We introduce Ragas (Retrieval Augmented Generation Assessment), a framework for reference-free evaluation of Retrieval Augmented Generation (RAG) pipelines. RAG systems are compose…
Systematic Relational Reasoning With Epistemic Graph Neural Networks
Irtaza Khalid, Steven Schockaert
Developing models that can learn to reason is a notoriously challenging problem. We focus on reasoning in relational domains, where the use of Graph Neural Networks (GNNs) seems li…