2 citations · 2 across the 15 of their papers we have counts for
7 papers · 1 filter
gLSTM: Mitigating Over-Squashing by Increasing Storage Capacity
Hugh Blayney, Álvaro Arroyo, Xiaowen Dong +1
Graph Neural Networks (GNNs) leverage the graph structure to transmit information between nodes, typically through the message-passing mechanism. While these models have found a wi…
Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin
Enrique Queipo-de-Llano, Álvaro Arroyo, Federico Barbero +4
Attention sinks and compression valleys have attracted significant attention as two puzzling phenomena in large language models, but have been studied in isolation. In this work, w…
On the Impact of Downstream Tasks on Sampling and Reconstructing Noisy Graph Signals
Baskaran Sripathmanathan, Xiaowen Dong, Michael Bronstein
We investigate graph signal reconstruction and sample selection for classification tasks. We present general theoretical characterisations of classification error applicable to mul…
Mathematical Foundations of Geometric Deep Learning
Haitz Sáez de Ocáriz Borde, Michael Bronstein
We review the key mathematical concepts necessary for studying Geometric Deep Learning.
Why do LLMs attend to the first token?
Federico Barbero, Álvaro Arroyo, Xiangming Gu +4
Large Language Models (LLMs) tend to attend heavily to the first token in the sequence -- creating a so-called attention sink. Many works have studied this phenomenon in detail, pr…
Towards Quantifying Long-Range Interactions in Graph Machine Learning: a Large Graph Dataset and a Measurement
Huidong Liang, Haitz Sáez de Ocáriz Borde, Baskaran Sripathmanathan +2
Long-range dependencies are critical for effective graph representation learning, yet most existing datasets focus on small graphs tailored to inductive tasks, offering limited ins…
On Vanishing Gradients, Over-Smoothing, and Over-Squashing in GNNs: Bridging Recurrent and Graph Learning
Álvaro Arroyo, Alessio Gravina, Benjamin Gutteridge +5
Graph Neural Networks (GNNs) are models that leverage the graph structure to transmit information between nodes, typically through the message-passing operation. While widely succe…