47 citations · 262 across the 33 of their papers we have counts for
4 papers · 1 filter
TRA: Better Length Generalisation with Threshold Relative Attention
Mattia Opper, Roland Fernandez, Paul Smolensky +1
Transformers struggle with length generalisation, displaying poor performance even on basic tasks. We test whether these limitations can be explained through two key failures of th…
Compositional Processing Emerges in Neural Networks Solving Math Problems
Jacob Russin, Roland Fernandez, Hamid Palangi +4
A longstanding question in cognitive science concerns the learning mechanisms underlying compositionality in human cognition. Humans can infer the structured relationships (e.g., g…
Discovering the Compositional Structure of Vector Representations with Role Learning Networks
Paul Soulos, Tom McCoy, Tal Linzen +1
How can neural networks perform so well on compositional tasks even though they lack explicit compositional representations? We use a novel analysis technique called ROLE to show t…
Enhancing the Transformer with Explicit Relational Encoding for Math Problem Solving
Imanol Schlag, Paul Smolensky, Roland Fernandez +3
We incorporate Tensor-Product Representations within the Transformer in order to better support the explicit representation of relation structure. Our Tensor-Product Transformer (T…