2 citations · 2 across the 15 of their papers we have counts for
16 papers
Graph Neural Networks Are Not Continuous Across Graph Resolutions
Christian Koke, Yuesong Shen, Abhishek Saroha +4
We show that contrary to conventional wisdom in the community, graph neural networks (GNNs) are not continuous with respect to all natural modes of graph convergence. As a result,…
A Mechanistic Analysis of Looped Reasoning Language Models
Hugh Blayney, Álvaro Arroyo, Johan Obando-Ceron +4
Reasoning has become a central capability in large language models. Recent research has shown that reasoning performance can be improved by looping an LLM's layers in the latent di…
Can Graph Foundation Models Generalize Over Architecture?
Benjamin Gutteridge, Michael Bronstein, Xiaowen Dong
Graph foundation models (GFMs) have recently attracted interest due to the promise of graph neural network (GNN) architectures that generalize zero-shot across graphs of arbitrary…
gLSTM: Mitigating Over-Squashing by Increasing Storage Capacity
Hugh Blayney, Álvaro Arroyo, Xiaowen Dong +1
Graph Neural Networks (GNNs) leverage the graph structure to transmit information between nodes, typically through the message-passing mechanism. While these models have found a wi…
Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin
Enrique Queipo-de-Llano, Álvaro Arroyo, Federico Barbero +4
Attention sinks and compression valleys have attracted significant attention as two puzzling phenomena in large language models, but have been studied in isolation. In this work, w…
On the Impact of Downstream Tasks on Sampling and Reconstructing Noisy Graph Signals
Baskaran Sripathmanathan, Xiaowen Dong, Michael Bronstein
We investigate graph signal reconstruction and sample selection for classification tasks. We present general theoretical characterisations of classification error applicable to mul…
Mathematical Foundations of Geometric Deep Learning
Haitz Sáez de Ocáriz Borde, Michael Bronstein
We review the key mathematical concepts necessary for studying Geometric Deep Learning.
Why do LLMs attend to the first token?
Federico Barbero, Álvaro Arroyo, Xiangming Gu +4
Large Language Models (LLMs) tend to attend heavily to the first token in the sequence -- creating a so-called attention sink. Many works have studied this phenomenon in detail, pr…
Towards Quantifying Long-Range Interactions in Graph Machine Learning: a Large Graph Dataset and a Measurement
Huidong Liang, Haitz Sáez de Ocáriz Borde, Baskaran Sripathmanathan +2
Long-range dependencies are critical for effective graph representation learning, yet most existing datasets focus on small graphs tailored to inductive tasks, offering limited ins…
On Vanishing Gradients, Over-Smoothing, and Over-Squashing in GNNs: Bridging Recurrent and Graph Learning
Álvaro Arroyo, Alessio Gravina, Benjamin Gutteridge +5
Graph Neural Networks (GNNs) are models that leverage the graph structure to transmit information between nodes, typically through the message-passing operation. While widely succe…
Enhancing the Expressivity of Temporal Graph Networks through Source-Target Identification
Benedict Aaron Tjandra, Federico Barbero, Michael Bronstein
Despite the successful application of Temporal Graph Networks (TGNs) for tasks such as dynamic node classification and link prediction, they still perform poorly on the task of dyn…
Scalable Message Passing Neural Networks: No Need for Attention in Large Graph Representation Learning
Haitz Sáez de Ocáriz Borde, Artem Lukoianov, Anastasis Kratsios +2
We propose Scalable Message Passing Neural Networks (SMPNNs) and demonstrate that, by integrating standard convolutional message passing into a Pre-Layer Normalization Transformer-…
Neural Spacetimes for DAG Representation Learning
Haitz Sáez de Ocáriz Borde, Anastasis Kratsios, Marc T. Law +2
We propose a class of trainable deep learning-based geometries called Neural Spacetimes (NSTs), which can universally represent nodes in weighted directed acyclic graphs (DAGs) as…
On the Impact of Sample Size in Reconstructing Noisy Graph Signals: A Theoretical Characterisation
Baskaran Sripathmanathan, Xiaowen Dong, Michael Bronstein
Reconstructing a signal on a graph from noisy observations of a subset of the vertices is a fundamental problem in the field of graph signal processing. This paper investigates how…
Bundle Neural Networks for message diffusion on graphs
Jacob Bamberger, Federico Barbero, Xiaowen Dong +1
The dominant paradigm for learning on graph-structured data is message passing. Despite being a strong inductive bias, the local message passing mechanism suffers from pathological…
Can strong structural encoding reduce the importance of Message Passing?
Floor Eijkelboom, Erik Bekkers, Michael Bronstein +1
The most prevalent class of neural networks operating on graphs are message passing neural networks (MPNNs), in which the representation of a node is updated iteratively by aggrega…