584 citations · 688 across the 17 of their papers we have counts for
14 papers · 1 filter
Olmo 3
Team Olmo, :, Allyson Ettinger +66
We introduce Olmo 3, a family of state-of-the-art, fully-open language models at the 7B and 32B parameter scales. Olmo 3 model construction targets long-context reasoning, function…
How Language Model Hallucinations Can Snowball
Muru Zhang, Ofir Press, William Merrill +2
A major risk of using language models in practical applications is their tendency to hallucinate incorrect statements. Hallucinations are often attributed to knowledge gaps in LMs,…
Transparency Helps Reveal When Language Models Learn Meaning
Zhaofeng Wu, William Merrill, Hao Peng +2
Many current NLP systems are built from language models trained to optimize unsupervised objectives on large amounts of raw text. Under what conditions might such a procedure acqui…
Entailment Semantics Can Be Extracted from an Ideal Language Model
William Merrill, Alex Warstadt, Tal Linzen
Language models are often trained on text alone, without additional grounding. There is debate as to how much of natural language semantics can be inferred from such a procedure. W…
Saturated Transformers are Constant-Depth Threshold Circuits
William Merrill, Ashish Sabharwal, Noah A. Smith
Transformers have become a standard neural network architecture for many NLP problems, motivating theoretical analysis of their power in terms of formal languages. Recent work has…
Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand?
William Merrill, Yoav Goldberg, Roy Schwartz +1
Language models trained on billions of tokens have recently led to unprecedented results on many NLP tasks. This success raises the question of whether, in principle, a system can…
Competency Problems: On Finding and Removing Artifacts in Language Data
Matt Gardner, William Merrill, Jesse Dodge +4
Much recent work in NLP has documented dataset artifacts, bias, and spurious correlations between input features and output labels. However, how to tell which features have "spurio…
Formal Language Theory Meets Modern NLP
William Merrill
NLP is deeply intertwined with the formal study of language, both conceptually and historically. Arguably, this connection goes all the way back to Chomsky's Syntactic Structures i…
A Formal Hierarchy of RNN Architectures
William Merrill, Gail Weiss, Yoav Goldberg +3
We develop a formal hierarchy of the expressive capacity of RNN architectures. The hierarchy is based on two formal properties: space complexity, which measures the RNN's memory, a…
On the Linguistic Capacity of Real-Time Counter Automata
William Merrill
Counter machines have achieved a newfound relevance to the field of natural language processing (NLP): recent work suggests some strong-performing recurrent neural networks utilize…
Detecting Syntactic Change Using a Neural Part-of-Speech Tagger
William Merrill, Gigi Felice Stark, Robert Frank
We train a diachronic long short-term memory (LSTM) part-of-speech tagger on a large corpus of American English from the 19th, 20th, and 21st centuries. We analyze the tagger's abi…
Finding Syntactic Representations in Neural Stacks
William Merrill, Lenny Khazan, Noah Amsel +3
Neural network architectures have been augmented with differentiable stacks in order to introduce a bias toward learning hierarchy-sensitive regularities. It has, however, proven d…
Sequential Neural Networks as Automata
William Merrill
This work attempts to explain the types of computation that neural networks can perform by relating them to automata. We first define what it means for a real-time network with bou…
End-to-end Graph-based TAG Parsing with Neural Networks
Jungo Kasai, Robert Frank, Pauli Xu +2
We present a graph-based Tree Adjoining Grammar (TAG) parser that uses BiLSTMs, highway connections, and character-level CNNs. Our best end-to-end parser, which jointly performs su…