5 papers · 1 filter
Uncovering Latent Reasoning Strategies in Language Models
Awni Altabaa, John Lafferty
A language model trained on reasoning tasks learns to solve problems via multiple distinct strategies, yet these strategies are implicit and entangled within the m…
Unlocking Out-of-Distribution Generalization in Transformers via Recursive Latent Space Reasoning
Awni Altabaa, Siyu Chen, John Lafferty +1
Systematic, compositional generalization beyond the training distribution remains a core challenge in machine learning -- and a critical bottleneck for the emergent reasoning abili…
Disentangling and Integrating Relational and Sensory Information in Transformer Architectures
Awni Altabaa, John Lafferty
Relational reasoning is a central component of generally intelligent systems, enabling robust and data-efficient inductive generalization. Recent empirical evidence shows that many…
Learning Hierarchical Relational Representations through Relational Convolutions
Awni Altabaa, John Lafferty
An evolving area of research in deep learning is the study of architectures and inductive biases that support the learning of relational feature representations. In this paper, we…
Approximation of relation functions and attention mechanisms
Awni Altabaa, John Lafferty
Inner products of neural network feature maps arise in a wide variety of machine learning frameworks as a method of modeling relations between inputs. This work studies the approxi…