4 papers
Perturbation: A simple and efficient adversarial tracer for representation learning in language models
Joshua Rozner, Cory Shain
Linguistic representation learning in deep neural language models (LMs) has been studied for decades, for both practical and theoretical reasons. However, finding representations i…
BabyLM's First Constructions: Causal probing provides a signal of learning
Joshua Rozner, Leonie Weissweiler, Cory Shain
Construction grammar posits that language learners acquire constructions (form-meaning pairings) from the statistics of their environment. Recent work supports this hypothesis by s…
Constructions are Revealed in Word Distributions
Joshua Rozner, Leonie Weissweiler, Kyle Mahowald +1
Construction grammar posits that constructions, or form-meaning pairings, are acquired through experience with language (the distributional learning hypothesis). But how much infor…
Decrypting Cryptic Crosswords: Semantically Complex Wordplay Puzzles as a Target for NLP
Josh Rozner, Christopher Potts, Kyle Mahowald
Cryptic crosswords, the dominant crossword variety in the UK, are a promising target for advancing NLP systems that seek to process semantically complex, highly compositional langu…