31 citations · 38 across the 9 of their papers we have counts for
10 papers
Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin
Enrique Queipo-de-Llano, Álvaro Arroyo, Federico Barbero +4
Attention sinks and compression valleys have attracted significant attention as two puzzling phenomena in large language models, but have been studied in isolation. In this work, w…
On Vanishing Gradients, Over-Smoothing, and Over-Squashing in GNNs: Bridging Recurrent and Graph Learning
Álvaro Arroyo, Alessio Gravina, Benjamin Gutteridge +5
Graph Neural Networks (GNNs) are models that leverage the graph structure to transmit information between nodes, typically through the message-passing operation. While widely succe…
Enhancing the Expressivity of Temporal Graph Networks through Source-Target Identification
Benedict Aaron Tjandra, Federico Barbero, Michael Bronstein
Despite the successful application of Temporal Graph Networks (TGNs) for tasks such as dynamic node classification and link prediction, they still perform poorly on the task of dyn…
Bundle Neural Networks for message diffusion on graphs
Jacob Bamberger, Federico Barbero, Xiaowen Dong +1
The dominant paradigm for learning on graph-structured data is message passing. Despite being a strong inductive bias, the local message passing mechanism suffers from pathological…
Locality-Aware Graph-Rewiring in GNNs
Federico Barbero, Ameya Velingker, Amin Saberi +2
Graph Neural Networks (GNNs) are popular models for machine learning on graphs that typically follow the message-passing paradigm, whereby the feature of a node is updated recursiv…
On Over-Squashing in Message Passing Neural Networks: The Impact of Width, Depth, and Topology
Francesco Di Giovanni, Lorenzo Giusti, Federico Barbero +3
Message Passing Neural Networks (MPNNs) are instances of Graph Neural Networks that leverage the graph to send messages over the edges. This inductive bias leads to a phenomenon kn…
Latent Graph Inference using Product Manifolds
Haitz Sáez de Ocáriz Borde, Anees Kazi, Federico Barbero +1
Graph Neural Networks usually rely on the assumption that the graph topology is available to the network as well as optimal for the downstream task. Latent graph inference allows m…
Graph Neural Network Expressivity and Meta-Learning for Molecular Property Regression
Haitz Sáez de Ocáriz Borde, Federico Barbero
We demonstrate the applicability of model-agnostic algorithms for meta-learning, specifically Reptile, to GNN models in molecular regression tasks. Using meta-learning we are able…
Sheaf Neural Networks with Connection Laplacians
Federico Barbero, Cristian Bodnar, Haitz Sáez de Ocáriz Borde +3
A Sheaf Neural Network (SNN) is a type of Graph Neural Network (GNN) that operates on a sheaf, an object that equips a graph with vector spaces over its nodes and edges and linear…
Transcending Transcend: Revisiting Malware Classification in the Presence of Concept Drift
Federico Barbero, Feargus Pendlebury, Fabio Pierazzi +1
Machine learning for malware classification shows encouraging results, but real deployments suffer from performance degradation as malware authors adapt their techniques to evade d…