15 citations · 40 across the 21 of their papers we have counts for
23 papers
The Universal Classifier for Graph Learning
Ben Finkelshtein, André Linhares, Petar Veličković +2
While foundation models have revolutionized natural language processing and computer vision by leveraging universal vocabularies, Graph Machine Learning (GML) remains fractured due…
The Computational Basis of Confidence in Large Language Models
Dharshan Kumaran, Viorica Patraucean, Maks Ovsjanikov +2
Reliable confidence -- the probability that a model's own answer is correct -- is essential for the trustworthy deployment of language models. Existing work has largely evaluated c…
How LLMs Detect and Correct Their Own Errors: The Role of Internal Confidence Signals
Dharshan Kumaran, Viorica Patraucean, Simon Osindero +2
Large language models can detect their own errors and sometimes correct them without external feedback, but the underlying mechanisms remain unknown. We investigate this through th…
Causal Evidence that Language Models use Confidence to Drive Behavior
Dharshan Kumaran, Nathaniel Daw, Simon Osindero +2
Metacognition -- assessing the quality of one's own cognitive performance -- guides adaptive behavior across species. Substantial research demonstrates that confidence signals can…
How do LLMs Compute Verbal Confidence
Dharshan Kumaran, Arthur Conmy, Federico Barbero +3
Verbal confidence -- prompting LLMs to state their confidence as a number or category -- is widely used to extract uncertainty estimates from black-box models. However, how LLMs in…
Leveraging Classical Algorithms for Graph Neural Networks
Jason Wu, Petar Veličković
Neural networks excel at processing unstructured data but often fail to generalise out-of-distribution, whereas classical algorithms guarantee correctness but lack flexibility. We…
Extracting alignment data in open models
Federico Barbero, Xiangming Gu, Christopher A. Choquette-Choo +6
In this work, we show that it is possible to extract significant amounts of alignment training data from a post-trained model -- useful to steer the model to improve certain capabi…
Optimizers Qualitatively Alter Solutions And We Should Leverage This
Razvan Pascanu, Clare Lyle, Ionut-Vlad Modoranu +6
Due to the nonlinear nature of Deep Neural Networks (DNNs), one can not guarantee convergence to a unique global minimum of the loss when using optimizers relying only on local inf…
How Overconfidence in Initial Choices and Underconfidence Under Criticism Modulate Change of Mind in Large Language Models
Dharshan Kumaran, Stephen M Fleming, Larisa Markeeva +8
Large language models (LLMs) exhibit strikingly conflicting behaviors: they can appear steadfastly overconfident in their initial answers whilst at the same time being prone to exc…
What makes a good feedforward computational graph?
Alex Vitvitskyi, João G. M. Araújo, Marc Lackenby +1
As implied by the plethora of literature on graph rewiring, the choice of computational graph employed by a neural network can make a significant impact on its downstream performan…
Amplifying human performance in combinatorial competitive programming
Petar Veličković, Alex Vitvitskyi, Larisa Markeeva +4
Recent years have seen a significant surge in complex AI systems for competitive programming, capable of performing at admirable levels against human competitors. While steady prog…
NAR-*ICP: Neural Execution of Classical ICP-based Pointcloud Registration Algorithms
Efimia Panagiotaki, Daniele De Martini, Lars Kunze +2
This study explores the intersection of neural networks and classical robotics algorithms through the Neural Algorithmic Reasoning (NAR) blueprint, enabling the training of neural…
Cayley Graph Propagation
JJ Wilson, Maya Bechler-Speicher, Petar Veličković
In spite of the plethora of success stories with graph neural networks (GNNs) on modelling graph-structured data, they are notoriously vulnerable to over-squashing, whereby tasks n…
Recurrent Aggregators in Neural Algorithmic Reasoning
Kaijia Xu, Petar Veličković
Neural algorithmic reasoning (NAR) is an emerging field that seeks to design neural networks that mimic classical algorithmic computations. Today, graph neural networks (GNNs) are…
Commute-Time-Optimised Graphs for GNNs
Igor Sterner, Shiye Su, Petar Veličković
We explore graph rewiring methods that optimise commute time. Recent graph rewiring approaches facilitate long-range interactions in sparse graphs, making such rewirings commute-ti…
Temporal Graph Rewiring with Expander Graphs
Katarina Petrović, Shenyang Huang, Farimah Poursafaei +1
Evolving relations in real-world networks are often modelled by temporal graphs. Temporal Graph Neural Networks (TGNNs) emerged to model evolutionary behaviour of such graphs by le…
Position: Categorical Deep Learning is an Algebraic Theory of All Architectures
Bruno Gavranović, Paul Lessard, Andrew Dudzik +3
We present our position on the elusive quest for a general-purpose framework for specifying and studying deep learning architectures. Our opinion is that the key attempts made so f…
Position: Topological Deep Learning is the New Frontier for Relational Learning
Theodore Papamarkou, Tolga Birdal, Michael Bronstein +19
Topological deep learning (TDL) is a rapidly evolving field that uses topological features to understand and design deep learning models. This paper posits that TDL is the new fron…
TacticAI: an AI assistant for football tactics
Zhe Wang, Petar Veličković, Daniel Hennes +20
Identifying key patterns of tactics implemented by rival teams, and developing effective responses, lies at the heart of modern football. However, doing so algorithmically remains…
A Generalist Neural Algorithmic Learner
Borja Ibarz, Vitaly Kurin, George Papamakarios +12
The cornerstone of neural algorithmic reasoning is the ability to solve algorithmic tasks, especially in a way that generalises out of distribution. While recent years have seen a…
The CLRS Algorithmic Reasoning Benchmark
Petar Veličković, Adrià Puigdomènech Badia, David Budden +5
Learning representations of algorithms is an emerging area of machine learning, seeking to bridge concepts from neural networks with classical algorithms. Several important works h…
Learning heuristics for A*
Danilo Numeroso, Davide Bacciu, Petar Veličković
Path finding in graphs is one of the most studied classes of problems in computer science. In this context, search algorithms are often extended with heuristics for a more efficien…
Message passing all the way up
Petar Veličković
The message passing framework is the foundation of the immense success enjoyed by graph neural networks (GNNs) in recent years. In spite of its elegance, there exist many problems…