6 papers
Length Generalization Needs Proper Regularization
Pavlo Vasylenko, Matthias Lindemann, André F. T. Martins +1
Length generalization is the ability of sequential models to perform well on context lengths unseen during training. In this work, we show that the challenge of achieving length ge…
How Well Do Vision-Language Models Understand Movies?
Emmanouil Zaranis, António Farinhas, Saul Santos +28
Vision-language models (VLMs) can increasingly handle extended contexts and answer questions about long videos--but can they understand the narrative structure of movies as humans…
Long-Context Generalization with Sparse Attention
Pavlo Vasylenko, Hugo Pitorro, André F. T. Martins +1
Transformer-based architectures traditionally employ softmax to compute attention weights, which produces dense distributions over all tokens in a sequence. While effective in many…
How Effective are State Space Models for Machine Translation?
Hugo Pitorro, Pavlo Vasylenko, Marcos Treviso +1
Transformers are the current architecture of choice for NLP, but their attention layers do not scale well to long contexts. Recent works propose to replace attention with linear re…
TopoBench: A Framework for Benchmarking Topological Deep Learning
Lev Telyatnikov, Guillermo Bernardez, Marco Montagna +34
This work introduces TopoBench, an open-source library designed to standardize benchmarking and accelerate research in topological deep learning (TDL). TopoBench decomposes TDL int…
Incorporating Graph Information in Transformer-based AMR Parsing
Pavlo Vasylenko, Pere-Lluís Huguet Cabot, Abelardo Carlos Martínez Lorenzo +1
Abstract Meaning Representation (AMR) is a Semantic Parsing formalism that aims at providing a semantic graph abstraction representing a given text. Current approaches are based on…