183 citations · 432 across the 25 of their papers we have counts for
3 papers · 1 filter
Adaptively Sparse Transformers
Gonçalo M. Correia, Vlad Niculae, André F. T. Martins
Attention mechanisms have become ubiquitous in NLP. Recent architectures, notably the Transformer, learn powerful context-aware word representations through layered, multi-headed a…
Sparse Sequence-to-Sequence Models
Ben Peters, Vlad Niculae, André F. T. Martins
Sequence-to-sequence models are a powerful workhorse of NLP. Most variants employ a softmax transformation in both their attention mechanism and output layer, leading to dense alig…
Learning with Fenchel-Young Losses
Mathieu Blondel, André F. T. Martins, Vlad Niculae
Over the past decades, numerous loss functions have been been proposed for a variety of supervised learning tasks, including regression, classification, ranking, and more generally…