4 papers · 1 filter
Scaling Graph Reasoning in Language Models with Learnable Graph Positional Encodings
Javier Porras-Valenzuela, Arush Arora, Alejandro Ribeiro
Solving reasoning tasks supported on graphs typically requires providing a Language Reasoning Model (LRM) with a serialized description of the graph. This approach results in the i…
Size-Transferability of Graph Transformers with Convolutional Positional Encodings
Javier Porras-Valenzuela, Zhiyang Wang, Xiaotao Shang +3
Transformers have achieved remarkable success across domains, motivating the rise of Graph Transformers (GTs) as attention-based architectures for graph-structured data. A key desi…
A Constrained Optimization Perspective of Unrolled Transformers
Javier Porras-Valenzuela, Samar Hadou, Alejandro Ribeiro
We introduce a constrained optimization framework for training transformers that behave like optimization descent algorithms. Specifically, we enforce layerwise descent constraints…
Loss Shaping Constraints for Long-Term Time Series Forecasting
Ignacio Hounie, Javier Porras-Valenzuela, Alejandro Ribeiro
Several applications in time series forecasting require predicting multiple steps ahead. Despite the vast amount of literature in the topic, both classical and recent deep learning…