2 papers
cs.CL2026
Predicting the Emergence of Induction Heads in Language Model Pretraining
Tatsuya Aoyama, Ethan Gotlieb Wilcox, Nathan Schneider
Specialized attention heads dubbed induction heads (IHs) have been argued to underlie the remarkable in-context learning capabilities of modern language models; yet, a precise char…
cs.CL2026
Speaking of Language: Reflections on Metalanguage Research in NLP
Nathan Schneider, Antonios Anastasopoulos
This work aims to shine a spotlight on the topic of metalanguage. We first define metalanguage, link it to NLP and LLMs, and then discuss our two labs' metalanguage-centered effort…