2 citations · 3 across the 7 of their papers we have counts for
21 papers
The Role of -gram Smoothing in the Age of Neural Networks
Luca Malagutti, Andrius Buinovskij, Anej Svete +3
For nearly three decades, language models derived from the -gram assumption held the state of the art on the task. The key to their success lay in the application of various smo…
An Exploration of Left-Corner Transformations
Andreas Opedal, Eleftheria Tsipidi, Tiago Pimentel +2
The left-corner transformation (Rosenkrantz and Lewis, 1970) is used to remove left recursion from context-free grammars, which is an important step towards making the grammar pars…
Schrödinger's Bat: Diffusion Models Sometimes Generate Polysemous Words in Superposition
Jennifer C. White, Ryan Cotterell
Recent work has shown that despite their impressive capabilities, text-to-image diffusion models such as DALL-E 2 (Ramesh et al., 2022) can display strange behaviours when a prompt…
The Architectural Bottleneck Principle
Tiago Pimentel, Josef Valvoda, Niklas Stoehr +1
In this paper, we seek to measure how much information a component in a neural network could extract from the representations fed into it. Our work stands in contrast to prior prob…
Mutual Information Alleviates Hallucinations in Abstractive Summarization
Liam van der Poel, Ryan Cotterell, Clara Meister
Despite significant progress in the quality of language generated from abstractive summarization models, these models still exhibit the tendency to hallucinate, i.e., output conten…
Estimating the Entropy of Linguistic Distributions
Aryaman Arora, Clara Meister, Ryan Cotterell
Shannon entropy is often a quantity of interest to linguists studying the communicative capacity of human language. However, entropy must typically be estimated from observed data…