82 citations · 156 across the 24 of their papers we have counts for
28 papers
Know When to Hold 'em: Correct-Token Retention in Uniform-State Diffusion Language Models
Mojtaba Nafez, James Henderson
Uniform-state diffusion models (USDMs) can revise any token at any denoising step, which lets them correct their own mistakes, a key advantage over masked diffusion. Self-correctio…
Nonparametric Variational Regularisation of Pretrained Transformers
Fabio Fehr, James Henderson
The current paradigm of large-scale pre-training and fine-tuning Transformer large language models has lead to significant improvements across the board in natural language process…
Transformers as Graph-to-Graph Models
James Henderson, Alireza Mohammadshahi, Andrei C. Coman +1
We argue that Transformers are essentially graph-to-graph models, with sequences just being a special case. Attention weights are functionally equivalent to graph edges. Our Graph-…
Learning to Abstract with Nonparametric Variational Information Bottleneck
Melika Behjati, Fabio Fehr, James Henderson
Learned representations at the level of characters, sub-words, words and sentences, have each contributed to advances in understanding different NLP tasks and linguistic phenomena.…
GADePo: Graph-Assisted Declarative Pooling Transformers for Document-Level Relation Extraction
Andrei C. Coman, Christos Theodoropoulos, Marie-Francine Moens +1
Document-level relation extraction typically relies on text-based encoders and hand-coded pooling heuristics to aggregate information learned by the encoder. In this paper, we leve…
TESS: Text-to-Text Self-Conditioned Simplex Diffusion
Rabeeh Karimi Mahabadi, Hamish Ivison, Jaesung Tae +4
Diffusion models have emerged as a powerful paradigm for generation, obtaining strong performance in various continuous domains. However, applying continuous diffusion models to na…