7 papers
Locally Typical Sampling
Clara Meister, Tiago Pimentel, Gian Wiher +1
Today's probabilistic language generators fall short when it comes to producing coherent and fluent text despite the fact that the underlying models perform well under standard met…
Testing the Predictions of Surprisal Theory in 11 Languages
Ethan Gotlieb Wilcox, Tiago Pimentel, Clara Meister +2
A fundamental result in psycholinguistics is that less predictable words take a longer time to process. One theoretical explanation for this finding is Surprisal Theory (Hale, 2001…
Towards a Similarity-adjusted Surprisal Theory
Clara Meister, Mario Giulianelli, Tiago Pimentel
Surprisal theory posits that the cognitive effort required to comprehend a word is determined by its contextual predictability, quantified as surprisal. Traditionally, surprisal th…
How to Compute the Probability of a Word
Tiago Pimentel, Clara Meister
Language models (LMs) estimate a probability distribution over strings in a natural language; these distributions are crucial for computing perplexity and surprisal in linguistics…
Investigating Critical Period Effects in Language Acquisition through Neural Language Models
Ionut Constantinescu, Tiago Pimentel, Ryan Cotterell +1
Humans appear to have a critical period (CP) for language acquisition: Second language (L2) acquisition becomes harder after early childhood, and ceasing exposure to a first langua…
Speakers Fill Lexical Semantic Gaps with Context
Tiago Pimentel, Rowan Hall Maudslay, Damián Blasi +1
Lexical ambiguity is widespread in language, allowing for the reuse of economical word forms and therefore making language more efficient. If ambiguous words cannot be disambiguate…