Showing 2024Show all
3 papers · 1 filter
cs.CL2024
From Language Models over Tokens to Language Models over Characters
Tim Vieira, Ben LeBrun, Mario Giulianelli +5
Modern language models are internally -- and mathematically -- distributions over strings rather than strings, posing numerous challenges for programm…
cs.CL2024
On the Proper Treatment of Tokenization in Psycholinguistics
Mario Giulianelli, Luca Malagutti, Juan Luis Gastaldi +3
Language models are widely used in computational psycholinguistics to test theories that relate the negative log probability (the surprisal) of a region of interest (a substring of…
cs.CL2024
PILA: A Historical-Linguistic Dataset of Proto-Italic and Latin
Stephen Bothwell, Brian DuSell, David Chiang +1
Computational historical linguistics seeks to systematically understand processes of sound change, including during periods at which little to no formal recording of language is at…