5 papers · 1 filter
N-vium: Mixture-of-Exits Transformer for Accelerated Exact Generation
Aleksander Lorenc, Frédéric Berdoz, Joël Mathys +1
Improving the inference efficiency of autoregressive transformers typically means reducing FLOPs per token, usually through approximations that degrade model quality. We introduce…
From Message-Passing to Linearized Graph Sequence Models
Joël Mathys, Basil Rohner, Saku Peltonen +1
Message-passing based approaches form the default backbone of most learning architectures on graph-structured data. However, the rapid progress of modern deep learning architecture…
Learn to Jump: Adaptive Random Walks for Long-Range Propagation through Graph Hierarchies
Joël Mathys, Federico Errica
Message-passing architectures struggle to sufficiently model long-range dependencies in node and graph prediction tasks. We propose a novel approach exploiting hierarchical graph s…
Beyond Interpolation: Extrapolative Reasoning with Reinforcement Learning and Graph Neural Networks
Niccolò Grillo, Andrea Toccaceli, Joël Mathys +3
Despite incredible progress, many neural architectures fail to properly generalize beyond their training distribution. As such, learning to reason in a correct and generalizable wa…
Flood and Echo Net: Algorithmically Aligned GNNs that Generalize
Joël Mathys, Florian Grötschla, Kalyan Varma Nadimpalli +1
Most Graph Neural Networks follow the standard message-passing framework where, in each step, all nodes simultaneously communicate with each other. We want to challenge this paradi…