124 citations · 171 across the 6 of their papers we have counts for
7 papers
RECKONING: Reasoning through Dynamic Knowledge Encoding
Zeming Chen, Gail Weiss, Eric Mitchell +2
Recent studies on transformer-based language models show that they can answer questions by reasoning over knowledge provided as part of the context (i.e., in-context reasoning). Ho…
Thinking Like Transformers
Gail Weiss, Yoav Goldberg, Eran Yahav
What is the computational model behind a Transformer? Where recurrent neural networks have direct parallels in finite state machines, allowing clear discussion and thought around a…
Synthesizing Context-free Grammars from Recurrent Neural Networks (Extended Version)
Daniel M. Yellin, Gail Weiss
We present an algorithm for extracting a subclass of the context free grammars (CFGs) from a trained recurrent neural network (RNN). We develop a new framework, pattern rule sets (…
A Formal Hierarchy of RNN Architectures
William Merrill, Gail Weiss, Yoav Goldberg +3
We develop a formal hierarchy of the expressive capacity of RNN architectures. The hierarchy is based on two formal properties: space complexity, which measures the RNN's memory, a…
Learning Deterministic Weighted Automata with Queries and Counterexamples
Gail Weiss, Yoav Goldberg, Eran Yahav
We present an algorithm for extraction of a probabilistic deterministic finite automaton (PDFA) from a given black-box language model, such as a recurrent neural network (RNN). The…
On the Practical Computational Power of Finite Precision RNNs for Language Recognition
Gail Weiss, Yoav Goldberg, Eran Yahav
While Recurrent Neural Networks (RNNs) are famously known to be Turing complete, this relies on infinite precision in the states and unbounded computation time. We consider the cas…
Extracting Automata from Recurrent Neural Networks Using Queries and Counterexamples
Gail Weiss, Yoav Goldberg, Eran Yahav
We present a novel algorithm that uses exact learning and abstraction to extract a deterministic finite automaton describing the state dynamics of a given trained RNN. We do this u…