4 papers · 1 filter
Minimization of Boolean Complexity in In-Context Concept Learning
Leroy Z. Wang, R. Thomas McCoy, Shane Steinert-Threlkeld
What factors contribute to the relative success and corresponding difficulties of in-context learning for Large Language Models (LLMs)? Drawing on insights from the literature on h…
The Impact of Syntactic and Semantic Proximity on Machine Translation with Back-Translation
Nicolas Guerin, Shane Steinert-Threlkeld, Emmanuel Chemla
Unsupervised on-the-fly back-translation, in conjunction with multilingual pretraining, is the dominant method for unsupervised neural machine translation. Theoretically, however,…
Embedding structure matters: Comparing methods to adapt multilingual vocabularies to new languages
C. M. Downey, Terra Blevins, Nora Goldfine +1
Pre-trained multilingual language models underpin a large portion of modern NLP tools outside of English. A strong baseline for specializing these models for specific languages is…
Evaluating Transformer's Ability to Learn Mildly Context-Sensitive Languages
Shunjie Wang, Shane Steinert-Threlkeld
Despite the fact that Transformers perform well in NLP tasks, recent studies suggest that self-attention is theoretically limited in learning even some regular and context-free lan…