4 papers
Language Models over Canonical Byte-Pair Encodings
Tim Vieira, Tianyu Liu, Clemente Pasti +7
Modern language models represent probability distributions over character strings as distributions over (shorter) token strings derived via a deterministic tokenizer, such as byte-…
Information Locality as an Inductive Bias for Neural Language Models
Taiga Someya, Anej Svete, Brian DuSell +3
Inductive biases are inherent in every machine learning system, shaping how models generalize from finite data. In the case of neural language models (LMs), debates persist as to w…
PILA: A Historical-Linguistic Dataset of Proto-Italic and Latin
Stephen Bothwell, Brian DuSell, David Chiang +1
Computational historical linguistics seeks to systematically understand processes of sound change, including during periods at which little to no formal recording of language is at…
Nondeterministic Stacks in Neural Networks
Brian DuSell
Human language is full of compositional syntactic structures, and although neural networks have contributed to groundbreaking improvements in computer systems that process language…