23 citations · 36 across the 3 of their papers we have counts for
5 papers
Thinking Like Transformers
Gail Weiss, Yoav Goldberg, Eran Yahav
What is the computational model behind a Transformer? Where recurrent neural networks have direct parallels in finite state machines, allowing clear discussion and thought around a…
Synthesizing Context-free Grammars from Recurrent Neural Networks (Extended Version)
Daniel M. Yellin, Gail Weiss
We present an algorithm for extracting a subclass of the context free grammars (CFGs) from a trained recurrent neural network (RNN). We develop a new framework, pattern rule sets (…
A Formal Hierarchy of RNN Architectures
William Merrill, Gail Weiss, Yoav Goldberg +3
We develop a formal hierarchy of the expressive capacity of RNN architectures. The hierarchy is based on two formal properties: space complexity, which measures the RNN's memory, a…
Learning Deterministic Weighted Automata with Queries and Counterexamples
Gail Weiss, Yoav Goldberg, Eran Yahav
We present an algorithm for extraction of a probabilistic deterministic finite automaton (PDFA) from a given black-box language model, such as a recurrent neural network (RNN). The…
On the Practical Computational Power of Finite Precision RNNs for Language Recognition
Gail Weiss, Yoav Goldberg, Eran Yahav
While Recurrent Neural Networks (RNNs) are famously known to be Turing complete, this relies on infinite precision in the states and unbounded computation time. We consider the cas…