Showing cs.CLShow all
3 papers · 1 filter
cs.CL2025
Long-Context Generalization with Sparse Attention
Pavlo Vasylenko, Hugo Pitorro, André F. T. Martins +1
Transformer-based architectures traditionally employ softmax to compute attention weights, which produces dense distributions over all tokens in a sequence. While effective in many…
cs.CL2024
How Effective are State Space Models for Machine Translation?
Hugo Pitorro, Pavlo Vasylenko, Marcos Treviso +1
Transformers are the current architecture of choice for NLP, but their attention layers do not scale well to long contexts. Recent works propose to replace attention with linear re…
cs.CL2023
Incorporating Graph Information in Transformer-based AMR Parsing
Pavlo Vasylenko, Pere-Lluís Huguet Cabot, Abelardo Carlos Martínez Lorenzo +1
Abstract Meaning Representation (AMR) is a Semantic Parsing formalism that aims at providing a semantic graph abstraction representing a given text. Current approaches are based on…