584 citations · 688 across the 17 of their papers we have counts for
6 papers · 1 filter
Transparency Helps Reveal When Language Models Learn Meaning
Zhaofeng Wu, William Merrill, Hao Peng +2
Many current NLP systems are built from language models trained to optimize unsupervised objectives on large amounts of raw text. Under what conditions might such a procedure acqui…
A Logic for Expressing Log-Precision Transformers
William Merrill, Ashish Sabharwal
One way to interpret the reasoning power of transformer-based language models is to describe the types of logical rules they can resolve over some input text. Recently, Chiang et a…
Entailment Semantics Can Be Extracted from an Ideal Language Model
William Merrill, Alex Warstadt, Tal Linzen
Language models are often trained on text alone, without additional grounding. There is debate as to how much of natural language semantics can be inferred from such a procedure. W…
The Parallelism Tradeoff: Limitations of Log-Precision Transformers
William Merrill, Ashish Sabharwal
Despite their omnipresence in modern NLP, characterizing the computational power of transformer neural nets remains an interesting open question. We prove that transformers whose a…
ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehension
Sanjay Subramanian, William Merrill, Trevor Darrell +3
Training a referring expression comprehension (ReC) model for a new visual domain requires collecting referring expressions, and potentially corresponding bounding boxes, for image…
Extracting Finite Automata from RNNs Using State Merging
William Merrill, Nikolaos Tsilivis
One way to interpret the behavior of a blackbox recurrent neural network (RNN) is to extract from it a more interpretable discrete computational model, like a finite state machine,…