4 papers
Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin
Enrique Queipo-de-Llano, Ãlvaro Arroyo, Federico Barbero +4
Attention sinks and compression valleys have attracted significant attention as two puzzling phenomena in large language models, but have been studied in isolation. In this work, w…
On Vanishing Gradients, Over-Smoothing, and Over-Squashing in GNNs: Bridging Recurrent and Graph Learning
Ãlvaro Arroyo, Alessio Gravina, Benjamin Gutteridge +5
Graph Neural Networks (GNNs) are models that leverage the graph structure to transmit information between nodes, typically through the message-passing operation. While widely succe…
Bundle Neural Networks for message diffusion on graphs
Jacob Bamberger, Federico Barbero, Xiaowen Dong +1
The dominant paradigm for learning on graph-structured data is message passing. Despite being a strong inductive bias, the local message passing mechanism suffers from pathological…
Enhancing the Expressivity of Temporal Graph Networks through Source-Target Identification
Benedict Aaron Tjandra, Federico Barbero, Michael Bronstein
Despite the successful application of Temporal Graph Networks (TGNs) for tasks such as dynamic node classification and link prediction, they still perform poorly on the task of dyn…