2 citations · 3 across the 6 of their papers we have counts for
7 papers
Transformers Learn Faster with Semantic Focus
Parikshit Ram, Kenneth L. Clarkson, Tim Klinger +2
Various forms of sparse attention have been explored to mitigate the quadratic computational and memory cost of the attention mechanism in transformers. We study sparse transformer…
Quantifying artificial intelligence through algorithmic generalization
Takuya Ito, Murray Campbell, Lior Horesh +2
The rapid development of artificial intelligence (AI) systems has created an urgent need for their scientific quantification. While their fluency across a variety of domains is imp…
Neural Reasoning Networks: Efficient Interpretable Neural Networks With Automatic Textual Explanations
Stephen Carrow, Kyle Harper Erwin, Olga Vilenskaia +5
Recent advances in machine learning have led to a surge in adoption of neural networks for various tasks, but lack of interpretability remains an issue for many others in which an…
Learning interpretable positional encodings in transformers depends on initialization
Takuya Ito, Luca Cocchi, Tim Klinger +3
In transformers, the positional encoding (PE) provides essential information that distinguishes the position and order amongst tokens in a sequence. Most prior investigations of PE…
What makes Models Compositional? A Theoretical View: With Supplement
Parikshit Ram, Tim Klinger, Alexander G. Gray
Compositionality is thought to be a key component of language, and various compositional benchmarks have been developed to empirically probe the compositional generalization of exi…
EXPLORER: Exploration-guided Reasoning for Textual Reinforcement Learning
Kinjal Basu, Keerthiram Murugesan, Subhajit Chaudhury +3
Text-based games (TBGs) have emerged as an important collection of NLP tasks, requiring reinforcement learning (RL) agents to combine natural language understanding with reasoning.…
Compositional Program Generation for Few-Shot Systematic Generalization
Tim Klinger, Luke Liu, Soham Dan +3
Compositional generalization is a key ability of humans that enables us to learn new concepts from only a handful examples. Neural machine learning models, including the now ubiqui…