3 citations · 4 across the 8 of their papers we have counts for
7 papers · 1 filter
A Mechanistic Analysis of Looped Reasoning Language Models
Hugh Blayney, Álvaro Arroyo, Johan Obando-Ceron +4
Reasoning has become a central capability in large language models. Recent research has shown that reasoning performance can be improved by looping an LLM's layers in the latent di…
k-Maximum Inner Product Attention for Graph Transformers and the Expressive Power of GraphGPS
Jonas De Schouwer, Haitz Sáez de Ocáriz Borde, Xiaowen Dong
Graph transformers have shown promise in overcoming limitations of traditional graph neural networks, such as oversquashing and difficulties in modeling long-range dependencies. Ho…
gLSTM: Mitigating Over-Squashing by Increasing Storage Capacity
Hugh Blayney, Álvaro Arroyo, Xiaowen Dong +1
Graph Neural Networks (GNNs) leverage the graph structure to transmit information between nodes, typically through the message-passing mechanism. While these models have found a wi…
Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin
Enrique Queipo-de-Llano, Álvaro Arroyo, Federico Barbero +4
Attention sinks and compression valleys have attracted significant attention as two puzzling phenomena in large language models, but have been studied in isolation. In this work, w…
Return of ChebNet: Understanding and Improving an Overlooked GNN on Long Range Tasks
Ali Hariri, Álvaro Arroyo, Alessio Gravina +6
ChebNet, one of the earliest spectral GNNs, has largely been overshadowed by Message Passing Neural Networks (MPNNs), which gained popularity for their simplicity and effectiveness…
Neural Latent Geometry Search: Product Manifold Inference via Gromov-Hausdorff-Informed Bayesian Optimization
Haitz Saez de Ocariz Borde, Alvaro Arroyo, Ismael Morales +2
Recent research indicates that the performance of machine learning models can be improved by aligning the geometry of the latent space with the underlying data structure. Rather th…
Gromov-Hausdorff Distances for Comparing Product Manifolds of Model Spaces
Haitz Saez de Ocariz Borde, Alvaro Arroyo, Ismael Morales +2
Recent studies propose enhancing machine learning models by aligning the geometric characteristics of the latent space with the underlying data structure. Instead of relying solely…