4 papers · 1 filter
What is the Best Sequence Length for BABYLM?
Suchir Salhan, Richard Diehl Martinez, Zébulon Goriely +1
Transformer language models typically operate with a fixed-length context window, which has grown in step with large-scale pretraining datasets. In the BabyLM Challenge, however, m…
ByteSpan: Information-Driven Subword Tokenisation
Zébulon Goriely, Suchir Salhan, Pietro Lesci +2
Recent dynamic tokenisation methods operate directly on bytes and pool their latent representations into patches. This bears similarities to computational models of word segmentati…
BabyLM's First Words: Word Segmentation as a Phonological Probing Task
Zébulon Goriely, Paula Buttery
Language models provide a key framework for studying linguistic theories based on prediction, but phonological analysis using large language models (LLMs) is difficult; there are f…
IPA-CHILDES & G2P+: Feature-Rich Resources for Cross-Lingual Phonology and Phonemic Language Modeling
Zébulon Goriely, Paula Buttery
In this paper, we introduce two resources: (i) G2P+, a tool for converting orthographic datasets to a consistent phonemic representation; and (ii) IPA CHILDES, a phonemic dataset o…