3 citations · 3 across the 2 of their papers we have counts for
7 papers · 1 filter
Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand?
William Merrill, Yoav Goldberg, Roy Schwartz +1
Language models trained on billions of tokens have recently led to unprecedented results on many NLP tasks. This success raises the question of whether, in principle, a system can…
Formal Language Theory Meets Modern NLP
William Merrill
NLP is deeply intertwined with the formal study of language, both conceptually and historically. Arguably, this connection goes all the way back to Chomsky's Syntactic Structures i…
A Formal Hierarchy of RNN Architectures
William Merrill, Gail Weiss, Yoav Goldberg +3
We develop a formal hierarchy of the expressive capacity of RNN architectures. The hierarchy is based on two formal properties: space complexity, which measures the RNN's memory, a…
Detecting Syntactic Change Using a Neural Part-of-Speech Tagger
William Merrill, Gigi Felice Stark, Robert Frank
We train a diachronic long short-term memory (LSTM) part-of-speech tagger on a large corpus of American English from the 19th, 20th, and 21st centuries. We analyze the tagger's abi…
Finding Syntactic Representations in Neural Stacks
William Merrill, Lenny Khazan, Noah Amsel +3
Neural network architectures have been augmented with differentiable stacks in order to introduce a bias toward learning hierarchy-sensitive regularities. It has, however, proven d…
Sequential Neural Networks as Automata
William Merrill
This work attempts to explain the types of computation that neural networks can perform by relating them to automata. We first define what it means for a real-time network with bou…