4 papers
Target-Language Generation in Multilingual Models: Activation Steering and Optimal Control
James A. Michaelov, Carmen Amo Alonso, Tyler A. Chang +1
Ensuring that multilingual language models generate coherent text in a specific target language is a major issue in multilingual language modeling. We develop an optimal control me…
BabAR: from phoneme recognition to developmental measures of young children's speech production
Marvin Lavechin, Elika Bergelson, Roger Levy
Studying early speech development at scale requires automatic tools, yet automatic phoneme recognition, especially for young children, remains largely unsolved. Building on decades…
Language Model Behavioral Phases are Consistent Across Architecture, Training Data, and Scale
James A. Michaelov, Roger P. Levy, Benjamin K. Bergen
We show that across architecture (Transformer vs. Mamba vs. RWKV), training dataset (OpenWebText vs. The Pile), and scale (14 million parameters to 12 billion parameters), autoregr…
Multimodal Input Aids a Bayesian Model of Phonetic Learning
Sophia Zhi, Roger P. Levy, Stephan C. Meylan
One of the many tasks facing the typically-developing child language learner is learning to discriminate between the distinctive sounds that make up words in their native language.…